-- Logs begin at Wed 2022-06-15 13:48:11 UTC, end at Wed 2022-06-15 15:05:04 UTC. -- Jun 15 14:28:23.179397 dv-cinder-843277-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 14:28:23.186669 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 14:28:23 2022] *** Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: nodename: dv-cinder-843277-3-iscsi Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: machine: x86_64 Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: clock source: unix Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: pcre jit disabled Jun 15 14:28:23.188688 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: detected number of CPU cores: 4 Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: current working directory: / Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: detected binary path: /usr/bin/uwsgi-core Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: your processes number limit is 47680 Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: your memory page size is 4096 bytes Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: detected max file descriptor number: 2048 Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: lock engine: pthread robust mutexes Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: thunder lock: enabled Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 14:28:23.189197 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 14:28:23.235448 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: Python main interpreter initialized at 0x55eb1ae529c0 Jun 15 14:28:23.235448 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: python threads support enabled Jun 15 14:28:23.235448 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: your server socket listen backlog is limited to 100 connections Jun 15 14:28:23.235448 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: your mercy for graceful operations on workers is 90 seconds Jun 15 14:28:23.235830 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 14:28:23.235912 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: *** Operational MODE: preforking *** Jun 15 14:28:23.236173 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: *** uWSGI is running in multiple interpreter mode *** Jun 15 14:28:23.236173 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: spawned uWSGI master process (pid: 80369) Jun 15 14:28:23.236405 dv-cinder-843277-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 14:28:23.237436 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: spawned uWSGI worker 1 (pid: 80378, cores: 1) Jun 15 14:28:23.237436 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: spawned uWSGI worker 2 (pid: 80379, cores: 1) Jun 15 14:28:23.237436 dv-cinder-843277-3-iscsi devstack@keystone.service[80369]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 14:28:28.837572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:28.837572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Jun 15 14:28:28.838364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:28.838364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Jun 15 14:28:28.845058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:28.845058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: from cryptography.utils import int_from_bytes Jun 15 14:28:28.845517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:28:28.845517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: from cryptography.utils import int_from_bytes Jun 15 14:28:28.887575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:28:28.887770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:28:28.887969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] command line args: [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:28:28.888249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:28:28.888434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ================================================================================ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:28:28.888693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.889006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_dir = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.889135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.889321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_source = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.889503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] debug = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.889723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.890227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.890313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.890536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.890790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.891026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.891210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_config_append = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.891393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.891620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_dir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.891752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.891990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.892247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.892503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.892757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.892938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.893119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.893367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.893619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.893872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:28:28.893997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.894122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:28:28.894233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.894347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] command line args: [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:28:28.894469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.894581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:28:28.894695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.894823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ================================================================================ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:28:28.894947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.895057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.895180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] admin_token = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.895338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.895717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config_dir = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.895834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.895961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] publish_errors = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config_source = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] debug = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.896948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] standard_threads = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.897916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_config_append = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_dir = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_journal = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_json = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.898864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_stderr = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_syslog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.899768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.899884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.900004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.900004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.900218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.900331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.900461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.900587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.900702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.900815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.900929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.901041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.901207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.901330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.901451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.901854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.901983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.902099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.902226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.902342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.902456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.902570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.902690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.902690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.902911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.903028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.903141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] publish_errors = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.903253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.903378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.903492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.903612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.903726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.903839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.903965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.904093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.904093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.904312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.904426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] standard_threads = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.904544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.904664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.904787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.904902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.905015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.905136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.905256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.905376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_journal = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.905376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_json = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.905598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.905732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_stderr = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.905847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.905961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_syslog = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.906074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.906201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:28:28.906317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.906437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.906551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.906665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.906784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.906965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.907913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.908970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.909938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.910051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.910171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.910285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.910930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.911891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.912911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.913970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.914908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.915956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.916977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917484 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.917982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.918936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.919983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.920899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.921889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.922956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.923652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.924391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.924584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.924584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.924801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.924922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.925904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.926032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.external = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.926815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.926949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.password = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.token = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.927987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.928926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.929996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.930946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.931969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.external = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.932955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.password = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.933968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.token = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.934909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.935979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.936928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937289 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.937884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.938968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.939973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.940923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.941900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.942990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.943944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.944883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.945973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.946920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.947983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.948929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.949886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.950960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.951920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.952035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.952150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.952537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.952657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.952772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.952888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953749 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.953993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.954919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.955983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.956928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.957981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.958892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.959954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960659 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.960890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.961998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.962890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.963928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.964884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.965953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.966910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.967905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.968959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.969930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.970853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.971983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.972824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.973444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.973563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.973765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.973885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.974932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.975982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.976808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:28:28.977026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.977026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.977026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.977351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.977468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.977749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.977992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.978231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.978466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.978714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.978968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.979197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.979441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.979745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.980008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.980254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.980502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.980740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.980981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.981218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.981455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.981714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.981953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.982197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.982432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.982676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.982910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.983163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.983398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.983642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.983876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.984118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.984354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:28:28.984527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:28:29.095247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.095437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.095620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.100874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.101110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.101334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.104658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.109848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.132947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.133178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.137009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.137244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.187348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.188773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.236673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.236909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.239606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.239841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.303733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.303962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.304189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.304412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.304634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.304783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.304870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.305024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.305249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.305470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.305842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.305927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.328974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.329815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.341206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.341395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.354400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.354582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.354811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.355987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.356212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.356431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.356649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:28:29.372735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.372906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.372992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:29.373118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:29.373271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.373451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.373538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:29.373653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:28:29.373941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.374898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.385790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.386930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.395313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.396070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.398927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.399210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.399475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.399560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.399754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.399846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.400902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401353 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.401912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.402175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.428741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.430192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.430402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.430683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.431814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.432094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.435965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.436253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.436539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.437510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.437898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.438196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.444694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.444981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.448277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.448277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.449487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.449807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.450423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.450423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.450637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.450913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.451187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.451459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.451542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.451755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.452037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.452573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.452845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.453115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.453380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.453646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.470979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.471262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.474747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.475028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.475914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.479553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.479642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.479876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.483158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.483452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.484996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.488517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.488718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.492507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.492607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.492839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.493124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.493407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.493706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.493989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.496215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.496512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.496795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.497077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.497352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.497630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.513497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.513815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.514091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.514364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.517797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.518084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.518350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.518613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.522977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.524691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.524978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.527585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.528051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.528344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.529309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.529690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.530058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.530365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.530669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.532709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.532999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.534708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.535013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.535301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.543937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.544219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.544510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.544795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.545078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.545355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.545631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.548956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.549237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.549518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.549822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.550100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.550369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.550640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.563655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.567564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.570102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.570932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.572536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.573330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.574350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.574633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.574928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.576632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.578511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.580291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.580578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.580855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.583051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.584679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.587933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.588225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.588510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.588789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.589126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.590769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.594083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.594372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.594654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.594942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.598397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.598691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.598948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.604716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.604960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.605238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.606814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.607099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.607381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.607660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.607933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.608222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.608498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.608783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.609063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.609429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.609732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.610015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.610562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.613020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:28:29.613301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.613652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.614357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.614648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.614941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.615217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.615487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.615763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.616129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.616407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.616688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.616964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.617225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:28:29.649352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55eb1ae529c0 pid: 80378 (default app) Jun 15 14:28:29.652776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c180cc-c360-4a5d-af5e-aef7a08fd2b7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:29.653076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c180cc-c360-4a5d-af5e-aef7a08fd2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:29.653359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c180cc-c360-4a5d-af5e-aef7a08fd2b7 None None] PATH_INFO: `/v3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:29.657184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55eb1ae529c0 pid: 80379 (default app) Jun 15 14:28:29.657592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Wed Jun 15 14:28:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:37.424706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.425278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.425278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.430015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:37.435245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:37.435677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:37.435805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:37.506109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: INFO dbcounter [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] Registered counter for database keystone Jun 15 14:28:37.527391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG oslo_db.sqlalchemy.engines [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80378) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:28:37.533477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writer thread running {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:28:37.710784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80378) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:28:37.711138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80378) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:28:37.765374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:37.766992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:37.889612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:28:38.028341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:38.101489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: INFO dbcounter [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] Registered counter for database keystone Jun 15 14:28:38.120834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG oslo_db.sqlalchemy.engines [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80379) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:28:38.126905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writer thread running {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:28:38.250054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4347c3-fede-47a0-ae85-6f0c3e9cee88 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:38.254549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:38.255227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.255517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.255799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.264235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:38.315759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING keystone.common.rbac_enforcer.enforcer [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:28:38.316077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06c097-a2ee-426e-92ed-60ca2d852aa5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:38.325646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Wed Jun 15 14:28:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:28:38.332521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:38.349799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ece315-f500-410d-a974-5c72bde6a44a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:38.354156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:38.355307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:38.355600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:38.355883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] PATH_INFO: `/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:38.357396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] RBAC: Authorizing `identity:get_project(project_id=10e451bce05f4f3c975f7e8a5cfb1440)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:38.408475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:28:38.408819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8283f5-2e86-410f-b976-fa4216087589 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:38.414183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Wed Jun 15 14:28:38 2022] GET /identity/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:40.509558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df213e18-d519-4e94-8bff-00742bdf7f39 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.510123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df213e18-d519-4e94-8bff-00742bdf7f39 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.510215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df213e18-d519-4e94-8bff-00742bdf7f39 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.514255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:40.519745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.520276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.520362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.538772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:40.540305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:40.548144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:28:40.571483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df213e18-d519-4e94-8bff-00742bdf7f39 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:40.583692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df213e18-d519-4e94-8bff-00742bdf7f39 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:40.587872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80c5b9-3f81-47d6-8d8e-68af59c60c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:40.588506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c5b9-3f81-47d6-8d8e-68af59c60c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.588793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c5b9-3f81-47d6-8d8e-68af59c60c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.589073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80c5b9-3f81-47d6-8d8e-68af59c60c36 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.596668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80c5b9-3f81-47d6-8d8e-68af59c60c36 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:40.598574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80c5b9-3f81-47d6-8d8e-68af59c60c36 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:40.607219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Wed Jun 15 14:28:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:28:40.612645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:40.622377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ecbec-7aa6-4bf6-8caf-53a34a4aeaa7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:40.626828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbc173-1c46-4146-b5ab-acef2453bece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:40.627693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbc173-1c46-4146-b5ab-acef2453bece admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:40.627978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbc173-1c46-4146-b5ab-acef2453bece admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:40.628279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbc173-1c46-4146-b5ab-acef2453bece admin admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:40.630327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbc173-1c46-4146-b5ab-acef2453bece admin admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:40.632152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbc173-1c46-4146-b5ab-acef2453bece admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:40.633738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Wed Jun 15 14:28:40 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:42.842852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3c39d-65b3-45f5-aff4-1b9609028870 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.843388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3c39d-65b3-45f5-aff4-1b9609028870 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.843485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3c39d-65b3-45f5-aff4-1b9609028870 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.844786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:42.849956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.850395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.850657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.865560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6441-2269-4144-aff2-643df464f7fa None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.865885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6441-2269-4144-aff2-643df464f7fa None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.866172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6441-2269-4144-aff2-643df464f7fa None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.867394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:42.867499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:42.868624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:42.872433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:42.872730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:42.873017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:42.913179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG passlib.handlers.bcrypt [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80379) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:28:42.913610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG passlib.handlers.bcrypt [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80379) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:28:42.950676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:42.959811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:42.961390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:42.973907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:42.997620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.061936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:43.079678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad18073-c6fb-4699-ac7b-9be15404219d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.084302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.114635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.115372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.115654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.115941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.126817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:43.138064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:43.138967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:28:43.139789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:43.145491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.186415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cd947-0298-4b4e-ae53-6a8c0abb576d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.188774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.226161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.226465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.227359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.227652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.227941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.244381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbb996-c048-49a0-a23c-3c1203f63a81 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.279143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.279827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.280113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.280674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.281143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:43.281455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.282889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.295336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:43.300780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.301087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.302369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.398834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:28:43.399234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347d2cb-0cc1-4e7d-8898-3a6e8db93e16 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.404233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.406948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.432422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.433308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.433601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.433919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.435131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.436590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.438245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:43.444320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.461956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.504821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cda4b6-89ca-4723-8238-f0f24107f6c2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.539581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.540378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.540672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.540953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.546855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eab05-7f1e-4791-99f8-5148c80072cc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.558979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.560466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.580617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.581432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:28:43.581991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:43.583034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:43.584028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:28:43.584028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:28:43.584028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:43.584028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:28:43.584293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.584293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.584440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:43.585267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.586851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.588633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:43.588886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.593854 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:43.645792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.672471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:43.692784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc99cd-8c16-4a08-aa44-5757d9e72310 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.693718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4066f00-fb76-4c2a-9c33-fbd176007372 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:43.727349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.727528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:43.728018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.728110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:43.728312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.728401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:43.728597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.728682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:43.729332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:43.729425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.domain.id=default, user.id=1ec9e7f1acc8440b837c5cce9d23e7c5) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:43.729646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.729765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:43.731062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ab90-2cee-4e40-aa00-7a3155913045 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.731202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a79f2-c0c8-49d6-8d83-f435070e6546 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:43.738140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/role_assignments?user.id=1ec9e7f1acc8440b837c5cce9d23e7c5&scope.domain.id=default&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:43.749289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:46.156148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c652cd1-4b9b-4ba1-a41b-d31179442b04 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.157059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c652cd1-4b9b-4ba1-a41b-d31179442b04 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.157625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c652cd1-4b9b-4ba1-a41b-d31179442b04 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.159226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:46.166498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.167307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.167766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.197624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:46.204183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:46.249419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f044e2-a4b1-4588-80a7-13863afc3cf1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.250030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f044e2-a4b1-4588-80a7-13863afc3cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.250495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f044e2-a4b1-4588-80a7-13863afc3cf1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.252060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 13/23] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:28:46.275153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.275707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.276168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.308601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:46.311773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:46.449967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 11/24] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:46.494394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:46.513686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:46.555038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:46.606114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:46.691502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d374-5849-44f1-9c04-fb3bddeb91ad None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.705962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:46.756966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b27a-cd72-4726-9f4a-82da2f94e5ed None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:46.767271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.768439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.769012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.769499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.808802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:46.810665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:46.822707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:46.823561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:46.823861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:46.824153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:46.843223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:46.858964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:46.859877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:28:46.860630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 15/26] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:28:46.867061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:46.867901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:46.868698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:28:46.868698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:28:46.868698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:28:46.868698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:28:46.868895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:46.870719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 12/27] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:28:46.883005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.097689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260d6f9-32fa-4f95-bf0c-6e7627f8c37b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.135294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.135985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.136281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.137403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.137403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:47.137690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.139124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5769db-97c8-4f8c-b7f7-b9da06053391 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.140729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.157841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 16/28] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:47.159078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346177-5c7e-49bd-9aed-21da08f08277 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.163390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047bb1-4684-4abd-aaf7-3748c9301f71 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.163510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.163654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047bb1-4684-4abd-aaf7-3748c9301f71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.163866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047bb1-4684-4abd-aaf7-3748c9301f71 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.165359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 17/29] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:47.171841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f8fcd-27a0-4e3d-9700-983171a0062a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.172116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f8fcd-27a0-4e3d-9700-983171a0062a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.172349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f8fcd-27a0-4e3d-9700-983171a0062a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.178303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 18/30] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:47.179160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f8fcd-27a0-4e3d-9700-983171a0062a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.225087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.225810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.226105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.226519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.227263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:47.227574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.229232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326630dc-fe8d-460e-a45c-afcaa311935a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.248892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 13/31] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:47.250574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.250869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.251152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.274287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:47.275486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:47.309524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f8fcd-27a0-4e3d-9700-983171a0062a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.348580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f07175-ecc3-4051-910e-78d639c97e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.349959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f07175-ecc3-4051-910e-78d639c97e69 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.350502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f07175-ecc3-4051-910e-78d639c97e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.350996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f07175-ecc3-4051-910e-78d639c97e69 None admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.352503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f07175-ecc3-4051-910e-78d639c97e69 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.354335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f07175-ecc3-4051-910e-78d639c97e69 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.360989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 19/32] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:47 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:47.362837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.363366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.363851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.367719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 14/33] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:47.374489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.387600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:47.389257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:47.455546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.466757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 20/34] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:47.467718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.521346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.535204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6990a-1967-4955-83e0-7efbb37165c1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.572213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.573002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.573356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.573734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.574754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.576263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.581216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6cc01-901c-4789-b8ae-3491845d0417 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.618794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.619523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.619818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.620106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.621317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.622804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.624592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 21/35] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:47.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.699079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.734358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 15/36] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:47.735795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:47.740610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6978e-cc7a-4f87-bd83-8e4cf3d41c7b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.780707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59ab16-baad-4726-bae3-f82d879b7ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.781406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59ab16-baad-4726-bae3-f82d879b7ec4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.781726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59ab16-baad-4726-bae3-f82d879b7ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.782016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59ab16-baad-4726-bae3-f82d879b7ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.782805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59ab16-baad-4726-bae3-f82d879b7ec4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.784218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59ab16-baad-4726-bae3-f82d879b7ec4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.882498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c7304-c50e-42aa-be2f-d457c36e8998 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:47.924154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:47.924875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:47.925750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:47.926045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:47.926834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:47.928459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:47.939730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:47.957724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 16/37] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/roles => generated 225 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:47.957724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.042069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 22/38] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Jun 15 14:28:47 2022] POST /identity/v3/roles => generated 219 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:48.043904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279419-88c6-4500-9dba-c95a89e04ef2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.044205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279419-88c6-4500-9dba-c95a89e04ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.044493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279419-88c6-4500-9dba-c95a89e04ef2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.045991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 23/39] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:48.047505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bfb5d-978f-4731-acae-4bfac4310c5e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.047807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bfb5d-978f-4731-acae-4bfac4310c5e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.048092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bfb5d-978f-4731-acae-4bfac4310c5e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.049522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 24/40] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:48.059413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0e7ba-5a68-414e-a611-4723162d4314 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.060000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0e7ba-5a68-414e-a611-4723162d4314 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.060420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0e7ba-5a68-414e-a611-4723162d4314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.080807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-38f0e7ba-5a68-414e-a611-4723162d4314 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:48.083000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-38f0e7ba-5a68-414e-a611-4723162d4314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:48.191904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 25/41] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:48.193650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.193983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.194338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.205815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74839a-c31e-4f07-ba4a-58c15f5fb0ce None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.215216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:48.216476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:48.256383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.257345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.257576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.257894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] PATH_INFO: `/v3/domains/default/users/1ec9e7f1acc8440b837c5cce9d23e7c5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.262037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1ec9e7f1acc8440b837c5cce9d23e7c5, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.263525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.277317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.289686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 17/42] 10.222.0.57 () {58 vars in 1361 bytes} [Wed Jun 15 14:28:47 2022] PUT /identity/v3/domains/default/users/1ec9e7f1acc8440b837c5cce9d23e7c5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:28:48.289686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.303011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 26/43] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:48.330002 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.370893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.421127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfda47-db08-47e2-8c15-e2689487fc9c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.459646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3224b1-9b70-406b-a290-9bc0c4fd37d5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.469986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.471044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.471351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.471643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.472878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.474767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.493715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 18/44] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Jun 15 14:28:48 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:48.496749 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778109d-de33-4c64-88be-ef0613900002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.497576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778109d-de33-4c64-88be-ef0613900002 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.497904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778109d-de33-4c64-88be-ef0613900002 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.498190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778109d-de33-4c64-88be-ef0613900002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.499444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778109d-de33-4c64-88be-ef0613900002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.501101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778109d-de33-4c64-88be-ef0613900002 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.502890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 27/45] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:48.509305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:48.625960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109b711-4837-4122-a2c2-f2a46f566461 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:48.647652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:48.666279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939f661-73a5-4c05-ad08-fc3041b5a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:48.667007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939f661-73a5-4c05-ad08-fc3041b5a720 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:48.667305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939f661-73a5-4c05-ad08-fc3041b5a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:48.667593 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939f661-73a5-4c05-ad08-fc3041b5a720 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:48.668331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939f661-73a5-4c05-ad08-fc3041b5a720 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:48.669820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939f661-73a5-4c05-ad08-fc3041b5a720 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:48.698803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 19/46] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:28:48 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:50.438197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168008d-83af-4a59-b9e2-9b9502558ea0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.438714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168008d-83af-4a59-b9e2-9b9502558ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.438802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168008d-83af-4a59-b9e2-9b9502558ea0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.440130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 28/47] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:50.445652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.446390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.446689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.464543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:50.466089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:50.532518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 20/48] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:50.556266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168008d-83af-4a59-b9e2-9b9502558ea0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.607118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:50.657116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168008d-83af-4a59-b9e2-9b9502558ea0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:50.690427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d43954-4c57-45db-bd70-535aef0de761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:50.691323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d43954-4c57-45db-bd70-535aef0de761 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.691618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d43954-4c57-45db-bd70-535aef0de761 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.691903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d43954-4c57-45db-bd70-535aef0de761 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.702208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d43954-4c57-45db-bd70-535aef0de761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a3d43954-4c57-45db-bd70-535aef0de761 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:50.710597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:50.711419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:28:50.712174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 29/49] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:50.717714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.821733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34257dc1-892f-4f22-8ce5-5b210a5f57bf None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:50.823271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:50.823512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7350-92d7-43f1-909e-dfd46367066d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.823817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7350-92d7-43f1-909e-dfd46367066d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.824111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7350-92d7-43f1-909e-dfd46367066d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.825506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 30/50] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:50.830745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.831070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.831359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.848211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:50.849417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:50.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:50.861543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:50.861878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:50.862171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:50.864061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:50.864382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:50.865787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:50.877056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 21/51] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:50.883123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.915376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 31/52] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:50.939432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:50.980331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:50.983662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e28efb-8af4-48af-b046-2f487e4609a4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.017133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.018060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.018355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.018686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.019967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.021409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.023084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 22/53] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:50 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:51.028405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.041589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb209-6e0f-4a63-898e-74c0e43889f6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.059885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.075210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.076013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.076311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.076592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.078096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.079562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.081198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 32/54] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:51.086765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.130897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bd840-0146-4287-ae3c-0a58c5a3ec19 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.163795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.164985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.165358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.165906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.167437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.169204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.171377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 23/55] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:51.179397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:51.187880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c483a56-0725-4dd1-bc52-bfadd469e4f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.198118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.224734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc9e66-f3c8-43bc-8b28-801935e1e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.225130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9e66-f3c8-43bc-8b28-801935e1e639 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.225427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9e66-f3c8-43bc-8b28-801935e1e639 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.225743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9e66-f3c8-43bc-8b28-801935e1e639 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.226467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc9e66-f3c8-43bc-8b28-801935e1e639 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.227932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc9e66-f3c8-43bc-8b28-801935e1e639 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.254053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 33/56] 10.222.0.57 () {60 vars in 1140 bytes} [Wed Jun 15 14:28:51 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:28:51.288237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:51.291780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b227-d04f-457f-83b0-4223186e772b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:51.329225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:51.329947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:51.330248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:51.330540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:51.331906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.domain.id=default, user.id=1ec9e7f1acc8440b837c5cce9d23e7c5) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:51.332234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:51.333773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b25e9c-4682-496f-889c-411c3b3e416c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:51.343050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 24/57] 10.222.0.57 () {56 vars in 1322 bytes} [Wed Jun 15 14:28:51 2022] GET /identity/v3/role_assignments?user.id=1ec9e7f1acc8440b837c5cce9d23e7c5&scope.domain.id=default&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 530 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:28:53.389869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85c5d-19cb-40f4-91cd-1eb7bb9313ee None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.390424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85c5d-19cb-40f4-91cd-1eb7bb9313ee None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.390424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85c5d-19cb-40f4-91cd-1eb7bb9313ee None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.391815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 34/58] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:53.396861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.397264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.397440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.414615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:53.415818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:53.485320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 25/59] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:53.509523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85c5d-19cb-40f4-91cd-1eb7bb9313ee None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.610562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85c5d-19cb-40f4-91cd-1eb7bb9313ee None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.642638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e6184-de1b-4437-a5aa-4b782041888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.643369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6184-de1b-4437-a5aa-4b782041888b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.644319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6184-de1b-4437-a5aa-4b782041888b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.644762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6184-de1b-4437-a5aa-4b782041888b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.645990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e6184-de1b-4437-a5aa-4b782041888b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:53.647442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e6184-de1b-4437-a5aa-4b782041888b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:53.649040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 35/60] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:53.654515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:53.744831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:53.751667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d57ad8-fec9-4899-ad8f-5036d2aa3690 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:53.785190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc52cb4-811f-4182-87e3-614d0b030743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:53.785883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc52cb4-811f-4182-87e3-614d0b030743 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:53.786179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc52cb4-811f-4182-87e3-614d0b030743 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:53.786467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc52cb4-811f-4182-87e3-614d0b030743 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:53.787182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc52cb4-811f-4182-87e3-614d0b030743 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:53.788636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc52cb4-811f-4182-87e3-614d0b030743 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:53.814022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 26/61] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:53 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:57.031233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b2e3-e58a-4a7c-9d09-cac0736ac237 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.032177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b2e3-e58a-4a7c-9d09-cac0736ac237 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.032668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b2e3-e58a-4a7c-9d09-cac0736ac237 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.034774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 36/62] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:57.039820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ca7e-ea52-414f-9973-43afe07ad276 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.040655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ca7e-ea52-414f-9973-43afe07ad276 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.041118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ca7e-ea52-414f-9973-43afe07ad276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.066737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0959ca7e-ea52-414f-9973-43afe07ad276 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:57.068287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0959ca7e-ea52-414f-9973-43afe07ad276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:57.164512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 27/63] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:57.199911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14b2e3-e58a-4a7c-9d09-cac0736ac237 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.222746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10578-8622-4aee-961f-e8034fb7c99e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.223313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10578-8622-4aee-961f-e8034fb7c99e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.223764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10578-8622-4aee-961f-e8034fb7c99e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.225346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 28/64] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:57.230689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.231226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.231691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.237593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.255412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:57.256939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:57.365002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 29/65] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:28:57.374777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14b2e3-e58a-4a7c-9d09-cac0736ac237 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.413607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.423071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.427037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.429033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.429482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.430028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.458651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:57.466867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:57.467900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:57.468853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:28:57.470901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 37/66] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:57.476747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.565997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b0ee1-cae7-40d9-8711-a17e96cf3b76 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.600960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.604014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1beb4-0a0b-4674-bbb9-7b70a13076b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.605021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1beb4-0a0b-4674-bbb9-7b70a13076b9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.605454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1beb4-0a0b-4674-bbb9-7b70a13076b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.605917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1beb4-0a0b-4674-bbb9-7b70a13076b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.614543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89cbea-301a-4cd4-b0e9-ba681f56a2ec None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.619767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1beb4-0a0b-4674-bbb9-7b70a13076b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-72d1beb4-0a0b-4674-bbb9-7b70a13076b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:57.630260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:57.631288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:28:57.632139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 30/67] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:57.632812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e3d61-f3d9-435e-bc50-913c33f59010 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.633248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e3d61-f3d9-435e-bc50-913c33f59010 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.633737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e3d61-f3d9-435e-bc50-913c33f59010 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.635167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 31/68] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:57.636451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aaad8-39af-4204-a2cf-e515768bc6aa None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.636742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aaad8-39af-4204-a2cf-e515768bc6aa None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.637031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aaad8-39af-4204-a2cf-e515768bc6aa None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.638527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 32/69] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:57.639795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aaad8-39af-4204-a2cf-e515768bc6aa None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.659002 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.659699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.660003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.660293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.661059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:57.661530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.663087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f33da-9ce9-4425-bfab-795fc8e4e028 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.672729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 38/70] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:57.674402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590ad3d-194f-41b8-b39d-ede7b56ccb7d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.674742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590ad3d-194f-41b8-b39d-ede7b56ccb7d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.675031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590ad3d-194f-41b8-b39d-ede7b56ccb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.695798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0590ad3d-194f-41b8-b39d-ede7b56ccb7d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:57.697049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0590ad3d-194f-41b8-b39d-ede7b56ccb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:57.726084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.760904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aaad8-39af-4204-a2cf-e515768bc6aa None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.771020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 39/71] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:57.772331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd5d74-466f-4dc5-a193-c4197b90877a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.772634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd5d74-466f-4dc5-a193-c4197b90877a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.772923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd5d74-466f-4dc5-a193-c4197b90877a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.790038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-36fd5d74-466f-4dc5-a193-c4197b90877a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:57.791479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-36fd5d74-466f-4dc5-a193-c4197b90877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:57.796452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.797135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.797427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.797743 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.798505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:57.798822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.800387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.809704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 33/72] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:57.810516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.861177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 40/73] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:57.863380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f127-0336-46f1-bb4c-6609083e3577 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.863696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f127-0336-46f1-bb4c-6609083e3577 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.864006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f127-0336-46f1-bb4c-6609083e3577 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.865449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 41/74] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:28:57.870314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286f127-0336-46f1-bb4c-6609083e3577 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.886738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.912822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9b846-3437-4e83-a606-246a3be270cd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:57.941557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:57.946950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:57.947836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:57.948131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:57.948415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:57.963763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:57.965208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:57.966928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 34/75] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:57.967703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:57.979582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286f127-0336-46f1-bb4c-6609083e3577 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.014312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c8b1b-1d55-41e9-823f-01ada62f6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.015248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c8b1b-1d55-41e9-823f-01ada62f6149 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.015547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c8b1b-1d55-41e9-823f-01ada62f6149 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.015837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c8b1b-1d55-41e9-823f-01ada62f6149 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.027248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c8b1b-1d55-41e9-823f-01ada62f6149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a67c8b1b-1d55-41e9-823f-01ada62f6149 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:58.036261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:58.038005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:28:58.038763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 42/76] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:58.039454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.039454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.039574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.056032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:28:58.057236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:28:58.077255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4827e-0394-47b1-95cc-273c05318c66 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.111139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.112050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.112347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.112634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.113915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.115440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.117113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 35/77] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:57 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.118200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.122960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.127705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 43/78] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:28:58.128532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.188861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.226685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d65e1-75dd-4416-a326-a6540ec4f120 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.231027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60494efd-ce43-4d3c-ba9c-cf63212a78e3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.260458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.261273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.261565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.262004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.265418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.266264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.266561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.266850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.278017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:28:58.286525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:28:58.287473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:28:58.288263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 36/79] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:28:58.288916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.339393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.341945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.343918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.387427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0881b-3e5d-4b4d-a162-f0eb754edd03 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.403562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.421079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.421874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.422171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.422461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.423225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:58.423545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.425700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.433778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 37/80] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.434586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.454428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 44/81] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:58.454428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.538961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380675be-7e7c-4368-8490-7a28819d3326 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.552958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.555802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43727e-2b17-49cb-afa3-d51733e7ac13 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.573032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.573717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.574015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.574300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.589540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.590371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.590673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.590953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.592133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.593591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.595230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 45/82] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.595996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.628396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.648509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.649956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.695173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de405d-a66a-450a-ba77-6b22d911b088 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.730096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.730773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.731063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.731348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.732106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:58.732420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.733843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.742047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 46/83] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.746562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.748120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 38/84] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:58.748964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.775167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.829352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.845560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506e678-43a0-4292-973d-647293305cb6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.850099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bea08-c409-475a-b1fd-3214ff9384b7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.878311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.879192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.879494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.879778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.881040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.882534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.883995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:58.884205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 47/85] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.884869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:58.884966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.885161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:58.885443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:58.886660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:58.888096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:58.890025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 39/86] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:58.890669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:58.981799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:58.986114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be07253-020e-4232-9fd2-6133942e6d90 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:58.991367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665deca-99db-417f-bc1b-a23411633f7e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.020486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.021173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.021473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.021793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.022530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.024011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.025081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.025990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.026286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.026840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.028061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.029490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.031168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 40/87] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:28:58 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.031941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.050888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.053190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 48/88] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:28:58 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:28:59.054076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.154466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b22d-382c-4e8b-bb27-8d833eacdce3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.168328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74329fb2-c4d6-445d-80ac-e6bb38e872b1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.197993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.199049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.199373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.199691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.201085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.202945 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.205147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 41/89] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.206574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.213367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.214475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.214985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.215280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.245855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.304554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.306654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.308686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.333145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64eb21b-c2fd-4d5c-a459-b08acd6a483e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.376260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.376933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.377227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.378183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.379401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=892a6e9fd0e24095bfaeb9f4f5859115) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:59.379726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.381171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.387812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 42/90] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/role_assignments?user.id=892a6e9fd0e24095bfaeb9f4f5859115&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.388646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.443386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 49/91] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:59.444282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.510254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.520112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44a39-944f-4791-90f1-5b066624e871 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.564742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a18346-726b-4824-910e-a40485f83ad2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.566409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.567442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.567955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.568450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.605084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.605812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.606114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.606406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.607442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=1ec9e7f1acc8440b837c5cce9d23e7c5) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:28:59.607882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.609390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.609846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.616425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 50/92] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/role_assignments?user.id=1ec9e7f1acc8440b837c5cce9d23e7c5&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.617295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.707916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.709906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.830080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b3d2-d42b-440f-9932-f853485fbd72 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.833474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 43/93] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:28:59.840206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.870066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.899455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:28:59.905352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.906644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.907214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:28:59.907792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:28:59.909289 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:28:59.911203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:28:59.921931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 51/94] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:28:59.923273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:28:59.958886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53217459-e799-448f-a8fc-fa5ec0931a58 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:28:59.998061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:28:59.999356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:28:59.999947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.000243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] PATH_INFO: `/v3/projects/d376259b759a4ab79899015bc389247b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.001505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] RBAC: Authorizing `identity:get_project(project_id=d376259b759a4ab79899015bc389247b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.003866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.006059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 44/95] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/projects/d376259b759a4ab79899015bc389247b => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:00.012621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:00.124999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b5fe-14d3-4a34-81dd-b8153ca04859 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.163335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.164087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.164291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.164591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.165445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=d376259b759a4ab79899015bc389247b, user.id=892a6e9fd0e24095bfaeb9f4f5859115) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:00.166136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.166808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:00.170366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b433e5-7e94-4fe7-ae4d-d4d88090d1d4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.182239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 45/96] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:00 2022] GET /identity/v3/role_assignments?user.id=892a6e9fd0e24095bfaeb9f4f5859115&scope.project.id=d376259b759a4ab79899015bc389247b&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:00.206054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a1aad-ec8d-468b-832a-72821b86bf9d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:00.260131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:00.260822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:00.261119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:00.261410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:00.262304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] RBAC: Adding query filter params (role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=892a6e9fd0e24095bfaeb9f4f5859115) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:00.262632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:00.264169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e69ea4-7f58-4e6c-84aa-b820d4a4de91 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:00.270902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 52/97] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:28:59 2022] GET /identity/v3/role_assignments?user.id=892a6e9fd0e24095bfaeb9f4f5859115&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:01.778577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dd46b-bc46-4ffe-a68d-be810ed6906f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:01.779051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dd46b-bc46-4ffe-a68d-be810ed6906f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:01.779306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dd46b-bc46-4ffe-a68d-be810ed6906f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:01.780784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 46/98] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:01.786423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:01.786841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:01.787008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:01.804819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:01.806143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:01.855286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:01.878355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 53/99] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:01.901979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dd46b-bc46-4ffe-a68d-be810ed6906f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.011157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dd46b-bc46-4ffe-a68d-be810ed6906f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.046646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.047379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.047672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.047958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.059581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:02.067694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:02.068513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:02.069265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 47/100] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:02.074715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.177924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70d177-c5a2-4f0d-9e43-861b5cce92c4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.213547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.214267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.214563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.214849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.215622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:02.215942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.217352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.226603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 54/101] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:02.232156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.290135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.350841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad32e5-4ac9-48a3-ae46-5d0edd41b1fe None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.397329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d9cbb-1277-4195-abf1-3db691ced9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.398257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d9cbb-1277-4195-abf1-3db691ced9bd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.398556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d9cbb-1277-4195-abf1-3db691ced9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.398850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d9cbb-1277-4195-abf1-3db691ced9bd None admin] PATH_INFO: `/v3/users/1b489728ecd848ba94b4b1a2ed78bc3f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.418423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d9cbb-1277-4195-abf1-3db691ced9bd None admin] RBAC: Authorizing `identity:get_user(user_id=1b489728ecd848ba94b4b1a2ed78bc3f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.419932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d9cbb-1277-4195-abf1-3db691ced9bd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.421737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 48/102] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/users/1b489728ecd848ba94b4b1a2ed78bc3f => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:02.427364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.438623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545301b-d881-411b-a22c-cd918ad99a07 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.438926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545301b-d881-411b-a22c-cd918ad99a07 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.439247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545301b-d881-411b-a22c-cd918ad99a07 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.441292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 49/103] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:02.447185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.447496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.447791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.465413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:02.466659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:02.509538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.576858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 50/104] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:02.617279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25288-5f30-4609-a10c-f81dc4a39502 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.622751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.668539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.704797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080e536-f659-4e4b-b61b-819826f76119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.705513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080e536-f659-4e4b-b61b-819826f76119 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.705842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080e536-f659-4e4b-b61b-819826f76119 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.706187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080e536-f659-4e4b-b61b-819826f76119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.800092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16ba45-3810-4ab5-a2cf-99f2d8f972a2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:02.815774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080e536-f659-4e4b-b61b-819826f76119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:02.818019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080e536-f659-4e4b-b61b-819826f76119 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:02.845134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54857d-4306-413b-bd70-bae7cc1be44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:02.847848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54857d-4306-413b-bd70-bae7cc1be44d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.848454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54857d-4306-413b-bd70-bae7cc1be44d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.849993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54857d-4306-413b-bd70-bae7cc1be44d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.875441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54857d-4306-413b-bd70-bae7cc1be44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:02.886010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4e54857d-4306-413b-bd70-bae7cc1be44d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:02.886222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:02.887044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:02.887817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:02.889734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 51/105] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:02.891002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea712-0874-4e13-94c1-5ab5ad7bcc13 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.891301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea712-0874-4e13-94c1-5ab5ad7bcc13 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.891597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea712-0874-4e13-94c1-5ab5ad7bcc13 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:02.892905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 52/106] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:02.904369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea712-0874-4e13-94c1-5ab5ad7bcc13 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:02.993538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 55/107] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:02.995338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:02.996734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:02.998008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.025836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:03.028637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:03.064466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.097181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea712-0874-4e13-94c1-5ab5ad7bcc13 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.157701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.158429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.159294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.159727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.160582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:03.160972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.162602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec576e6-6827-4129-8d8c-31d8bf276c58 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.163626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 56/108] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:03.169369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.172639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 53/109] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.175074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626713f9-0142-4487-9d5a-1a39bce082da None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.175505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626713f9-0142-4487-9d5a-1a39bce082da None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.175915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626713f9-0142-4487-9d5a-1a39bce082da None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.177417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 54/110] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:03.182377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626713f9-0142-4487-9d5a-1a39bce082da None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.278046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.298090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e81bcd-33e0-4375-a9a8-ef9833fc0925 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.301641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626713f9-0142-4487-9d5a-1a39bce082da None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.335110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd104b39-08f8-4e84-9a2f-ff5b759bb0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.336237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104b39-08f8-4e84-9a2f-ff5b759bb0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.336626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104b39-08f8-4e84-9a2f-ff5b759bb0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.337012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd104b39-08f8-4e84-9a2f-ff5b759bb0b2 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.338582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104b39-08f8-4e84-9a2f-ff5b759bb0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.340311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd104b39-08f8-4e84-9a2f-ff5b759bb0b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.342565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 57/111] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.343359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.343970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.344358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.344469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.344651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.344894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.344997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.346269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.347918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.349749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 55/112] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:03.354441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.368739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:03.370009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:03.382780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.460288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 58/113] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:03.461846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.484143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69b6a2-2670-4975-802b-8dc841ec33da None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.524826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.526422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.527152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.527618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.529601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.531313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.534312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 56/114] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.535575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.556929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.602171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b4282-4c57-449e-a264-00f57551674e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.643898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.652742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.653513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.653891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.654263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.655265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] RBAC: Adding query filter params (role.id=27f359bbb37c4303b502646b2b96ca90, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=1b489728ecd848ba94b4b1a2ed78bc3f) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:03.655663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.657373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.662343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c77652-6ab9-41ed-937d-b64706f09664 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.665376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 59/115] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/role_assignments?user.id=1b489728ecd848ba94b4b1a2ed78bc3f&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=27f359bbb37c4303b502646b2b96ca90 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.666150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.714478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.715689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.716137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.716426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.717716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.719397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.726627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 57/116] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:03.727441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.792201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c034-78ae-48b8-878e-6fa2498a9c8d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.834984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778f6ac-0625-4c52-ba50-351b6713991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.835816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778f6ac-0625-4c52-ba50-351b6713991f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.836938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778f6ac-0625-4c52-ba50-351b6713991f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.836938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778f6ac-0625-4c52-ba50-351b6713991f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.838634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.848479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9e7b4-998e-4ed2-9047-da6cb9a74124 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:03.849868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778f6ac-0625-4c52-ba50-351b6713991f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5778f6ac-0625-4c52-ba50-351b6713991f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:03.858107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:03.858969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 60/117] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:03.865529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651169c-9e11-4937-b0a5-aa72cb96dcf7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.866404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651169c-9e11-4937-b0a5-aa72cb96dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.866404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651169c-9e11-4937-b0a5-aa72cb96dcf7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.866404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 61/118] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:03.866404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2daee-094a-4f5f-8c1b-81c40fe11b9e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.866404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2daee-094a-4f5f-8c1b-81c40fe11b9e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.866404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2daee-094a-4f5f-8c1b-81c40fe11b9e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.866720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 62/119] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:03.867120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2daee-094a-4f5f-8c1b-81c40fe11b9e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:03.898533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0522cb-d616-4171-83af-310a09bca5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:03.899214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0522cb-d616-4171-83af-310a09bca5df None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.899504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0522cb-d616-4171-83af-310a09bca5df None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.899788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0522cb-d616-4171-83af-310a09bca5df None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.900516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0522cb-d616-4171-83af-310a09bca5df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:03.901999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0522cb-d616-4171-83af-310a09bca5df None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:03.933890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 58/120] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:29:03 2022] POST /identity/v3/projects => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:03.935365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:03.935671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:03.935963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:03.955534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:03.961598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:03.963803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:04.016785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2daee-094a-4f5f-8c1b-81c40fe11b9e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.056731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e8b5e-e4e1-4f71-b7d8-c54e6c60968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.057239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 59/121] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:04.058078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8b5e-e4e1-4f71-b7d8-c54e6c60968a None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.058472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8b5e-e4e1-4f71-b7d8-c54e6c60968a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.058753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.059015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8b5e-e4e1-4f71-b7d8-c54e6c60968a None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/892a6e9fd0e24095bfaeb9f4f5859115/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.066566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e8b5e-e4e1-4f71-b7d8-c54e6c60968a None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7e424d403346c5bbf92a367142a6df, user_id=892a6e9fd0e24095bfaeb9f4f5859115, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:04.067627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e8b5e-e4e1-4f71-b7d8-c54e6c60968a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:04.093765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 63/122] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:03 2022] PUT /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/892a6e9fd0e24095bfaeb9f4f5859115/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:04.096889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.097332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.097627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.123270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:04.124591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:04.201464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:04.229188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f12384-1562-4ef8-a4f4-7525ba2b9553 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.234027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 64/123] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:04.234909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.294134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.294875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.295167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.295451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.296213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:04.296531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:04.297957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:04.301125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:04.309346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 60/124] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:04.310149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.395656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918037e-39d3-404f-b7d6-aabf7a67f058 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.481420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.482269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.482561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.482838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.501398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4824a-c4fc-4b22-86c5-0fe3f68395b2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.504624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:04.512399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:04.513329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:04.514133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:04.518217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 65/125] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:04.519302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.570110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:04.574754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.575579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.575864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.576149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.602291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:04.610288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:04.611142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:04.611948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 61/126] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:04.613611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.680121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:04.720351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca93b2d-c12e-46d8-b4d4-a3c1325ce4f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.727658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8335d89-a14d-42a7-b521-be8f6c01313e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.770442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.771733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.772028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.772318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.774629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.775363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.775653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.775932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.776700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:04.777024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:04.777870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:04.779345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:04.781255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 66/127] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:04.781819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.782881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:04.803178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 62/128] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:04.803881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:04.907227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b10c1a-512f-49cb-842e-f481765333d8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:04.922017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:04.943225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:04.943972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:04.944378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:04.944674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:04.945445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:04.945795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:04.947195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:04.956992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 67/129] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:04.957680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.067405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c41ae7-71de-41ed-9762-724bb4139f35 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.106172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.107196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.107493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.107778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.109334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.110849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.112863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 68/130] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:05.113485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.147639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9521a2-6c31-4362-b378-5e8dfd7e2428 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.175251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.194758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.216284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.217159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.217444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.217747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.219383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.220835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.223621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 63/131] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:04 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:05.224411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.304844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925037-58dd-4267-bc7f-677cbd866fc4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.375094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.376019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.376322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.376622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.378732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.380286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.382769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 69/132] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:05 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:05.385877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.391829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fabc17-267e-4573-a31c-48286c17fcb8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.437615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.438452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.438752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.439047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.440899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.442866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.445220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 64/133] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:05 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:05.448317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.526289 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea2d0-890f-4137-a917-492314cff4e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.543190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.574036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.592189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.593117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.593413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.593720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/892a6e9fd0e24095bfaeb9f4f5859115/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.602465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7e424d403346c5bbf92a367142a6df, user_id=892a6e9fd0e24095bfaeb9f4f5859115, role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.603954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.637507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dafde-f56c-4b9e-8d4f-ccc6023fbeee None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.643983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 70/134] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:05 2022] PUT /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/892a6e9fd0e24095bfaeb9f4f5859115/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:05.644815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.685683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.686815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.687127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.687438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] PATH_INFO: `/v3/projects/d376259b759a4ab79899015bc389247b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.688939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] RBAC: Authorizing `identity:get_project(project_id=d376259b759a4ab79899015bc389247b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.692261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.694305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 65/135] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:05 2022] GET /identity/v3/projects/d376259b759a4ab79899015bc389247b => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:05.700467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:05.837034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f227-567e-4366-8afd-a1d8ac21a28b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:05.882061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:05.901792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a4c0c-6572-49b8-b96b-7f98f3782640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:05.902971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a4c0c-6572-49b8-b96b-7f98f3782640 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:05.903331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a4c0c-6572-49b8-b96b-7f98f3782640 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:05.903640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a4c0c-6572-49b8-b96b-7f98f3782640 None admin] PATH_INFO: `/v3/projects/d376259b759a4ab79899015bc389247b/users/892a6e9fd0e24095bfaeb9f4f5859115/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:05.908412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a4c0c-6572-49b8-b96b-7f98f3782640 None admin] RBAC: Authorizing `identity:create_grant(project_id=d376259b759a4ab79899015bc389247b, user_id=892a6e9fd0e24095bfaeb9f4f5859115, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:05.910338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a4c0c-6572-49b8-b96b-7f98f3782640 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:05.935309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 66/136] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:05 2022] PUT /identity/v3/projects/d376259b759a4ab79899015bc389247b/users/892a6e9fd0e24095bfaeb9f4f5859115/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:05.984192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c781e-bf63-4ef1-b40f-be70e3e03808 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.033492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b70245-a9cd-4c34-96ae-2cd430f7bc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.034705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b70245-a9cd-4c34-96ae-2cd430f7bc32 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.034997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b70245-a9cd-4c34-96ae-2cd430f7bc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.035281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b70245-a9cd-4c34-96ae-2cd430f7bc32 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/1ec9e7f1acc8440b837c5cce9d23e7c5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.043329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b70245-a9cd-4c34-96ae-2cd430f7bc32 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7e424d403346c5bbf92a367142a6df, user_id=1ec9e7f1acc8440b837c5cce9d23e7c5, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:06.044836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b70245-a9cd-4c34-96ae-2cd430f7bc32 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:06.059927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 71/137] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:05 2022] PUT /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/1ec9e7f1acc8440b837c5cce9d23e7c5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:06.265546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360d163-fafc-432d-9140-f3b07b7e5f6b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.266149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360d163-fafc-432d-9140-f3b07b7e5f6b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.266149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360d163-fafc-432d-9140-f3b07b7e5f6b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.267466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 67/138] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:06.274353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.274672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.274937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.294772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:06.296014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:06.326399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:06.378889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 72/139] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:06.402520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360d163-fafc-432d-9140-f3b07b7e5f6b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.531563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360d163-fafc-432d-9140-f3b07b7e5f6b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.576444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a8a9d-b73b-4a3a-aa00-cca12a02e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.577513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8a9d-b73b-4a3a-aa00-cca12a02e33b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.577827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8a9d-b73b-4a3a-aa00-cca12a02e33b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.578110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8a9d-b73b-4a3a-aa00-cca12a02e33b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.591970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a8a9d-b73b-4a3a-aa00-cca12a02e33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f25a8a9d-b73b-4a3a-aa00-cca12a02e33b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:06.601228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:06.602177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:06.602955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 68/140] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:06.608597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.709690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d80f31-dd85-4e1b-97d9-571e9a00fecc None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.710019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d80f31-dd85-4e1b-97d9-571e9a00fecc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.710270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d80f31-dd85-4e1b-97d9-571e9a00fecc None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.712168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 69/141] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:06.718457 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.718754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.719041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.748335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:06.749533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:06.817927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0e0eb-3c2c-4b2d-949b-dd6057e1b435 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:06.821546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:06.861609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 70/142] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:06.903827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:06.904900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:06.905395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:06.905906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:06.906881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:06.907396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:06.909983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:06.926039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.926461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 73/143] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:06.931770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:06.982055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.092634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a000c7-6c48-4458-bf26-61b0e8d14e72 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.170977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3daf3a-fd51-4f24-8e45-93db16d96c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.171800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3daf3a-fd51-4f24-8e45-93db16d96c97 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.172086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3daf3a-fd51-4f24-8e45-93db16d96c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.172360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3daf3a-fd51-4f24-8e45-93db16d96c97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.175193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7dff7-da61-43bf-a31f-e957cbf386c5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.177302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3daf3a-fd51-4f24-8e45-93db16d96c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.178784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3daf3a-fd51-4f24-8e45-93db16d96c97 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.180418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 71/144] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:07.181601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3834b1e-2db7-403d-bbc6-ed4bc0ffb623 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.186900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3834b1e-2db7-403d-bbc6-ed4bc0ffb623 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.187197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3834b1e-2db7-403d-bbc6-ed4bc0ffb623 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.193892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 72/145] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:07.197222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3834b1e-2db7-403d-bbc6-ed4bc0ffb623 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.245219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adf76c-17ca-4278-a1c3-e6e91914cdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.246137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf76c-17ca-4278-a1c3-e6e91914cdde None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.246425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf76c-17ca-4278-a1c3-e6e91914cdde None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.246715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf76c-17ca-4278-a1c3-e6e91914cdde None admin] PATH_INFO: `/v3/users/1b489728ecd848ba94b4b1a2ed78bc3f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.251018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf76c-17ca-4278-a1c3-e6e91914cdde None admin] RBAC: Authorizing `identity:get_user(user_id=1b489728ecd848ba94b4b1a2ed78bc3f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.253161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf76c-17ca-4278-a1c3-e6e91914cdde None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.265047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 74/146] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:06 2022] GET /identity/v3/users/1b489728ecd848ba94b4b1a2ed78bc3f => generated 289 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:07.266478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.266770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.267054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.277458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.305700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:07.307078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:07.385475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 75/147] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:07.386327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.430405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.464006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3834b1e-2db7-403d-bbc6-ed4bc0ffb623 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.536874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.537605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.537941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.538228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.539021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.540492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.566604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da0e99-34c9-48bf-ae5d-47d56961bf1c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.577299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 73/148] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:07 2022] POST /identity/v3/users => generated 283 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:07.578255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.636723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.637530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.637851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.638134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.640111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:07.641712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:07.643927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 76/149] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:07.651306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.754410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.758872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188f004-8e75-45ba-9df5-89255f7f3830 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:07.821542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:07.823365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:07.823832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:07.824221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:07.847197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:07.859430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:07.860492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:07.861417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 74/150] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:07.869395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:07.917788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:07.953334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70136291-ccbd-4b5a-b4da-d4ca9f09d397 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.018465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296a5b-5407-40da-ae45-808476cc7a38 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.038134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.039045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.039336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.039618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/1b489728ecd848ba94b4b1a2ed78bc3f/roles/27f359bbb37c4303b502646b2b96ca90` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.043094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7e424d403346c5bbf92a367142a6df, user_id=1b489728ecd848ba94b4b1a2ed78bc3f, role_id=27f359bbb37c4303b502646b2b96ca90)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.044553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.060191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 77/151] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:07 2022] PUT /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df/users/1b489728ecd848ba94b4b1a2ed78bc3f/roles/27f359bbb37c4303b502646b2b96ca90 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:29:08.085027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.086210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.086752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.087266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.088369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:08.088942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.090810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.106435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 75/152] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.113083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.265818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca1078-b405-49fb-80b3-8a530f92381e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.308957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5cda6-84a8-4bbd-b6cf-cd30ca6adc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.309900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5cda6-84a8-4bbd-b6cf-cd30ca6adc8e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.310190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5cda6-84a8-4bbd-b6cf-cd30ca6adc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.310471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5cda6-84a8-4bbd-b6cf-cd30ca6adc8e None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.311763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5cda6-84a8-4bbd-b6cf-cd30ca6adc8e None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.313221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5cda6-84a8-4bbd-b6cf-cd30ca6adc8e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.317140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 78/153] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:08.322798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.351120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.469370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1edef8-9015-494e-80b9-2cb054b8421c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.511806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.512560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.512850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.513135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.602634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:08.604378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:08.630210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.654661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c85d0-aa1b-4008-a599-e592bc506fe7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.655165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c85d0-aa1b-4008-a599-e592bc506fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.655644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c85d0-aa1b-4008-a599-e592bc506fe7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.657195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 79/154] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:08.663143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.663625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.664055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.688080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:08.689489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:08.710055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:08.775673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 80/155] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:08.801798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.804776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 76/156] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:08.822636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:08.919135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c3007-8540-47f3-8c8b-49e8adc8396e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:08.958998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:08.959832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:08.960122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:08.960409 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:08.972527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:08.986412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:08.988024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:08.988876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:08.989473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 81/157] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:08.996067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.010026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.088507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12f5ce-8f45-403c-b7a8-951349cd20f1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.122788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.148797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.149708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.150000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.150281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.153123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.154924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.156771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 77/158] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.163833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.200523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ebae1-fba0-4ba0-838a-ca3a2201165c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.279414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.280410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.280732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.281019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.282145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.282492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.284076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0c777c-4d16-41e9-b625-e827c889ed9c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.301967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 82/159] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:09.305273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60666915-1fa7-44db-ab63-c6f240ef7792 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.305582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60666915-1fa7-44db-ab63-c6f240ef7792 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.305891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60666915-1fa7-44db-ab63-c6f240ef7792 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.307909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 83/160] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:09.311957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60666915-1fa7-44db-ab63-c6f240ef7792 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.350767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f539d7-1697-4f49-b5fc-b5562d196df7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.420386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.421059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.421344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.421622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.422501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=892a6e9fd0e24095bfaeb9f4f5859115) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:09.422821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.424923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d68d6-662f-411f-a43d-ceeb807c3242 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.434613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 78/161] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/role_assignments?user.id=892a6e9fd0e24095bfaeb9f4f5859115&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.435741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c72f-1653-4dab-9d35-2df78cc11ac4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.436021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c72f-1653-4dab-9d35-2df78cc11ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.436302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c72f-1653-4dab-9d35-2df78cc11ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.440284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.478785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-98e8c72f-1653-4dab-9d35-2df78cc11ac4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:09.479912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-98e8c72f-1653-4dab-9d35-2df78cc11ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:09.493438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60666915-1fa7-44db-ab63-c6f240ef7792 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.537129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367512-baa0-4cbd-b7b3-fcb6221f4ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.538954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367512-baa0-4cbd-b7b3-fcb6221f4ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.539752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367512-baa0-4cbd-b7b3-fcb6221f4ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.540234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367512-baa0-4cbd-b7b3-fcb6221f4ac1 None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.542049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20367512-baa0-4cbd-b7b3-fcb6221f4ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:09.543866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20367512-baa0-4cbd-b7b3-fcb6221f4ac1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:09.546203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 84/162] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:09.548619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301405d-434f-4739-b099-eeb3d8ce5214 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.549401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301405d-434f-4739-b099-eeb3d8ce5214 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.549936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301405d-434f-4739-b099-eeb3d8ce5214 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.552014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 85/163] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:09.557171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301405d-434f-4739-b099-eeb3d8ce5214 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.600429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.644398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 79/164] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:09.647436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.648806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.649299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.683932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:09.686442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:09.777973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 80/165] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:09.779365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:09.813686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301405d-434f-4739-b099-eeb3d8ce5214 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.912495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ef6a5-3f08-485e-ad65-1e30e5fb95be None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:09.938095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.952158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.953313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.953855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.954466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.965923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:09.970414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:09.971367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:09.971715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:09.972021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:09.975602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:09.976481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:09.977493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:09.980404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 81/166] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:09.981787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:09.982728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.097170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a1d91-9220-45b6-bd6e-6cdd819a35c9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.136445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.137879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.139063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.139558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.145308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.150951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.160186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:10.169570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:10.170503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:10.171296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:10.175054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 82/167] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:10.176301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.218140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.296709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837bef-4735-4beb-addf-f32c70d8874a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.334248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.335283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.335811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.336308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.337329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:10.337884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.339495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.348497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 83/168] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.349646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.496704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 86/169] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 940 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:10.497588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.498517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e6ea-c8d7-4a8e-814e-e03df02ba490 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.501962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.566408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.567115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.567411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.567695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.568474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:10.568790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.570228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.575982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.588035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 84/170] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.588900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.715791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c2942-f30e-4345-be3d-801a0d1e8b62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.730218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254701-e9df-4268-8025-05fcb65e2b96 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:10.765782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.767128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.767526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.767954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.769303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.770990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.772985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 85/171] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.773735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.810436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:10.831911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:10.832120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:10.832373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:10.832659 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] PATH_INFO: `/v3/users/892a6e9fd0e24095bfaeb9f4f5859115` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:10.838397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] RBAC: Authorizing `identity:get_user(user_id=892a6e9fd0e24095bfaeb9f4f5859115)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:10.839881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:10.841728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 87/172] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/users/892a6e9fd0e24095bfaeb9f4f5859115 => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:10.842356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:10.967278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821f140-6c9b-4e3e-a83c-9357a8823182 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.038123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ef0c7-a42b-443c-bb20-13c86e996fce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.042139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.043274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.043700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.044437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] PATH_INFO: `/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.044854 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.045860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec9e7f1acc8440b837c5cce9d23e7c5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.050958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.052666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 86/173] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/users/1ec9e7f1acc8440b837c5cce9d23e7c5 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:11.056310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.110732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.111475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.112327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.112614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.113462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] RBAC: Adding query filter params (role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=892a6e9fd0e24095bfaeb9f4f5859115) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:11.113804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.122693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.129960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 88/174] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:10 2022] GET /identity/v3/role_assignments?user.id=892a6e9fd0e24095bfaeb9f4f5859115&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:11.130894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.210413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.223872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a026bff-5614-48ea-8b95-9a51a3e615e4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.266618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.267641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.268746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.269188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.321791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc903b5-9b7d-40d6-b03a-4a6aa00a34eb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.385753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.386837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.387324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.387797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.422982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.422982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.466654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.525568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.528355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.614468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.632166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 87/175] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:11.637563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.725074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 89/176] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:11.731486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.841584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d69e7f-f178-42e5-803c-e65c092bcde4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.844823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:11.890932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55e93a-e7e1-46f5-a58c-113432bac734 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:11.947220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebaa0b-3604-4444-a9d7-3d90fde5a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.948137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebaa0b-3604-4444-a9d7-3d90fde5a259 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.948430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebaa0b-3604-4444-a9d7-3d90fde5a259 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.948715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebaa0b-3604-4444-a9d7-3d90fde5a259 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.950008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebaa0b-3604-4444-a9d7-3d90fde5a259 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.950404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:11.951302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.951756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebaa0b-3604-4444-a9d7-3d90fde5a259 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.951861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.952112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] PATH_INFO: `/v3/projects/d376259b759a4ab79899015bc389247b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.953402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] RBAC: Authorizing `identity:get_project(project_id=d376259b759a4ab79899015bc389247b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:11.953513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 90/177] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:11 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:11.954807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ab0cf-1e1c-4f59-a59c-3d68b5bc40b0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.955263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ab0cf-1e1c-4f59-a59c-3d68b5bc40b0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.955915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:11.956039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ab0cf-1e1c-4f59-a59c-3d68b5bc40b0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.957549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 91/178] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:11.961814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 88/179] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:11 2022] GET /identity/v3/projects/d376259b759a4ab79899015bc389247b => generated 312 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:11.963289 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:11.963794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:11.963901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:11.968134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:11.986117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:11.987554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:12.061089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 92/180] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:12.062058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.068504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.130008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610fe3f-d78b-496f-9e2f-589bd02ddc26 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.131258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.172942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78797ee-dfce-49ef-a86a-59be2f1035d1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.182675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.183353 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.183643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.184399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.185787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=1ec9e7f1acc8440b837c5cce9d23e7c5) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:12.186125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.187953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999570-8254-4123-a93f-dbbaa0e4ba1c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.198008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 89/181] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:11 2022] GET /identity/v3/role_assignments?user.id=1ec9e7f1acc8440b837c5cce9d23e7c5&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:12.199481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386f5-6871-45f6-ae16-c20f230e7210 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.200222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386f5-6871-45f6-ae16-c20f230e7210 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.200514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386f5-6871-45f6-ae16-c20f230e7210 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.202898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 90/182] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:12.204065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386f5-6871-45f6-ae16-c20f230e7210 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.223435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.224129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.224630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.224923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.225942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=d376259b759a4ab79899015bc389247b, user.id=892a6e9fd0e24095bfaeb9f4f5859115) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:12.226376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.227860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad9fe3-0db7-4694-a8d9-a8e9a4b610ba None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.241244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 93/183] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/role_assignments?user.id=892a6e9fd0e24095bfaeb9f4f5859115&scope.project.id=d376259b759a4ab79899015bc389247b&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:12.242655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.242959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.243247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.276556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:12.277822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:12.400862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386f5-6871-45f6-ae16-c20f230e7210 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.445323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 94/184] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:12.458320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.461434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.462367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.465683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.465683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.480061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.491942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:12.500018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:12.500918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:12.501737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:12.507120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 91/185] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:12.511893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.562875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.630537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f4a8f-f223-45a5-9234-e12e3e010f89 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.669503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8facb-24a4-46a3-bd8f-ab66b48903bc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.687016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fee112-648c-4578-93a1-01eb474aa1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.688475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fee112-648c-4578-93a1-01eb474aa1fd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.688926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fee112-648c-4578-93a1-01eb474aa1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.689351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fee112-648c-4578-93a1-01eb474aa1fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.706011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fee112-648c-4578-93a1-01eb474aa1fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-55fee112-648c-4578-93a1-01eb474aa1fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:12.714603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:12.715685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:12.716608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 95/186] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:12.717376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb715f7c-514f-4980-806e-d7b85cd7c88c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.717624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb715f7c-514f-4980-806e-d7b85cd7c88c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.718000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb715f7c-514f-4980-806e-d7b85cd7c88c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.719340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 96/187] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:12.720577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.722001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.722112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.722392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.723161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:12.723475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.724635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb715f7c-514f-4980-806e-d7b85cd7c88c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.724942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b098-056f-44e4-9d03-ae8c944eab2e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.735136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 92/188] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:12.736483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.737311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.737597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.757552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:12.758819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:12.801233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.825646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 93/189] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:12.826798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.827838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb715f7c-514f-4980-806e-d7b85cd7c88c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.849321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:12.864267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.864944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.865234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.865512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.866305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:12.866621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.868008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15b7c0-293a-49ff-9c88-0939c10de681 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.876899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 97/190] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:12.878504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc22d8d-ac94-4feb-878a-b0c656b02b49 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.878795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc22d8d-ac94-4feb-878a-b0c656b02b49 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.879076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc22d8d-ac94-4feb-878a-b0c656b02b49 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.880320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 98/191] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:12.881020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc22d8d-ac94-4feb-878a-b0c656b02b49 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:12.935803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33476fc-1453-4056-bf20-f4e77d6d98d6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.969218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:12.970126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:12.970412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:12.970689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] PATH_INFO: `/v3/users/18b2479f09cd4d5ca93b1eaae205daab` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:12.982322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc22d8d-ac94-4feb-878a-b0c656b02b49 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:12.984409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] RBAC: Authorizing `identity:get_user(user_id=18b2479f09cd4d5ca93b1eaae205daab)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:12.985889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:12.987674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 94/192] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/users/18b2479f09cd4d5ca93b1eaae205daab => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:12.988450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.015973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55000ef0-b2e3-4098-876b-c16fd42402e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.016787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55000ef0-b2e3-4098-876b-c16fd42402e9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.017085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55000ef0-b2e3-4098-876b-c16fd42402e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.017367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55000ef0-b2e3-4098-876b-c16fd42402e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.022775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.028464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55000ef0-b2e3-4098-876b-c16fd42402e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-55000ef0-b2e3-4098-876b-c16fd42402e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:13.037545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:13.038514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:13.039309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 99/193] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:13.040086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.040336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.040667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.057169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:13.058393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:13.092932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cad30-2b5d-441d-8c74-cb0947b9b3be None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.093534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.126196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 100/194] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:13.137854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.137854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.137854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.137854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] PATH_INFO: `/v3/users/1b489728ecd848ba94b4b1a2ed78bc3f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.138157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.138224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] RBAC: Authorizing `identity:get_user(user_id=1b489728ecd848ba94b4b1a2ed78bc3f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.138224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.138224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 95/195] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:12 2022] GET /identity/v3/users/1b489728ecd848ba94b4b1a2ed78bc3f => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.138224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.233409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4752831-bc07-440e-b14d-1c22577baff5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.234870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb8d0-a635-49eb-ae85-3ffacb6d4734 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.235008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.268543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.269204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.269492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.269805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.270395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.271010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.271294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.271569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.272486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:13.272801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.274946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.283064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 96/196] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.284445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.322594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.350456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.351947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.384657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219154-801c-41f2-ba71-53dd2021ac7d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.418295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.418969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.419283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.419566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.451130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 101/197] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:13.452257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.454651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.493538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.494950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.523908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.554561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdbf25-bfec-49e5-a8e0-27451cb86e27 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.594385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 97/198] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:13.594694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.595506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.595943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.596314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.596677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.598159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.600416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.604026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 102/199] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.604026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.660021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.697674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11e60e-4bb4-4276-984b-187f56126c24 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.704144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0688b-b189-465c-b63d-6a4c1652e993 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.730701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.731567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.731854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.732142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] PATH_INFO: `/v3/users/18b2479f09cd4d5ca93b1eaae205daab` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.733312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] RBAC: Authorizing `identity:get_user(user_id=18b2479f09cd4d5ca93b1eaae205daab)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.734788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.737123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 98/200] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/users/18b2479f09cd4d5ca93b1eaae205daab => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.737783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.738015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.738863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.739154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.739437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.740816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.743033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.744813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 103/201] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.745503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.756561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.839434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082101e-5d87-4f9a-9959-1fc054cef049 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.844453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11295c1-871d-43cc-8673-d46f80b7cbe6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:13.868447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.873959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.874865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.875136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.875416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.876589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.878748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.879617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:13.880292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:13.880449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 99/202] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:13.880871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:13.881154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:13.881251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.881899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:13.883629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:13.913279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 104/203] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:13 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:13.913279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:13.988454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:13.995553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5e7f-7917-41c6-bdec-891bc999be3a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.034380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.035178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.035475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.035756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.038215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96a4da-39c8-4988-84bc-bf73c8273454 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.078380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.079079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.079378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.079663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.080725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=18b2479f09cd4d5ca93b1eaae205daab) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:14.081524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.082742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.089124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 105/204] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/role_assignments?user.id=18b2479f09cd4d5ca93b1eaae205daab&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:14.089986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.130779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.133646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.136081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.254551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dc6a8-3a19-421a-9cc6-ccb09200bf15 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.289076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.290277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 100/205] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:14.297004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.301622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.302329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.302621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.302905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.303756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] RBAC: Adding query filter params (role.id=27f359bbb37c4303b502646b2b96ca90, scope.project.id=3b7e424d403346c5bbf92a367142a6df, user.id=1b489728ecd848ba94b4b1a2ed78bc3f) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:14.304075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.305524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.312896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 106/206] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/role_assignments?user.id=1b489728ecd848ba94b4b1a2ed78bc3f&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=27f359bbb37c4303b502646b2b96ca90 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:14.411546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:14.416090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358e203-ded5-45dd-899f-c0271fc45325 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.481093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b7d8c-3da6-42cf-ab43-f10762838e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.482326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7d8c-3da6-42cf-ab43-f10762838e33 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.482799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7d8c-3da6-42cf-ab43-f10762838e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.483221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7d8c-3da6-42cf-ab43-f10762838e33 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.486861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b7d8c-3da6-42cf-ab43-f10762838e33 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.488757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b7d8c-3da6-42cf-ab43-f10762838e33 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.490741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 101/207] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:14.496669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:14.619168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713478e-896f-47fa-a91a-68cef756b934 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:14.669193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:14.669193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:14.669193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:14.669193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:14.669783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] RBAC: Adding query filter params (role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=18b2479f09cd4d5ca93b1eaae205daab) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:14.669783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:14.671188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0891a-181c-45ea-90ad-041e1508cc45 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:14.679561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 107/208] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:14 2022] GET /identity/v3/role_assignments?user.id=18b2479f09cd4d5ca93b1eaae205daab&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:16.390001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08dfa-6a81-444b-b87a-1688e3f253ea None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.391053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08dfa-6a81-444b-b87a-1688e3f253ea None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.391543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08dfa-6a81-444b-b87a-1688e3f253ea None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.393734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 102/209] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:16.399297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07fb43-c184-43ed-a50d-1bd04e2bd978 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.399739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07fb43-c184-43ed-a50d-1bd04e2bd978 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.399917 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07fb43-c184-43ed-a50d-1bd04e2bd978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.416857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9c07fb43-c184-43ed-a50d-1bd04e2bd978 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:16.418113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9c07fb43-c184-43ed-a50d-1bd04e2bd978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:16.474721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.496922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 108/210] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:16.520850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e08dfa-6a81-444b-b87a-1688e3f253ea None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.545164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1f57b-f601-47c0-b162-8b6b6a2bc42d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.545466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1f57b-f601-47c0-b162-8b6b6a2bc42d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.545761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1f57b-f601-47c0-b162-8b6b6a2bc42d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.547918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 109/211] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:16.554668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcb520-d9b4-4be7-b3c5-1bd463e42294 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.555221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcb520-d9b4-4be7-b3c5-1bd463e42294 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.555702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcb520-d9b4-4be7-b3c5-1bd463e42294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.572818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-01bcb520-d9b4-4be7-b3c5-1bd463e42294 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:16.574044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-01bcb520-d9b4-4be7-b3c5-1bd463e42294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:16.641732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e08dfa-6a81-444b-b87a-1688e3f253ea None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.647641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 110/212] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:16.648954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce85472-643c-4818-ab5e-582ba47c2a53 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.649253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce85472-643c-4818-ab5e-582ba47c2a53 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.649536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce85472-643c-4818-ab5e-582ba47c2a53 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.650954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 111/213] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:16.656672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.656966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.657249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.677130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:16.678353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:16.691385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.692199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.692487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.692767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.704675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:16.713226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:16.714169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:16.714963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 103/214] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:16.715625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.754083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.754798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 112/215] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:16.755852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.804607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:16.850362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b02471-ba6f-4d1f-a238-592d8b7d95a5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.866240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9c4f7-bf09-4208-8706-a86075f4a3e2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:16.887874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.888680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.888970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.889247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.890459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:16.892072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:16.894423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 104/216] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:16.895122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.903175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:16.903873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.904162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.904464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.905921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:16.906263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:16.907729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862706da-b8bc-4424-928e-7dc97b1bc422 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:16.916758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 113/217] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:16.918039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a530e1-1e40-49ca-b973-4ff8d44c0cdd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:16.918334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a530e1-1e40-49ca-b973-4ff8d44c0cdd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:16.918613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a530e1-1e40-49ca-b973-4ff8d44c0cdd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:16.919852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 114/218] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:16.922459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a530e1-1e40-49ca-b973-4ff8d44c0cdd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:16.997554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.004228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e3605-f98e-41c8-a6dd-cb3b1becb0a8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.023983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a530e1-1e40-49ca-b973-4ff8d44c0cdd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.032853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.040102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.040907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.041200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.041485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.051450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:17.057524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0885f07-796c-4bda-a095-9f40f821d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.058248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0885f07-796c-4bda-a095-9f40f821d8fb None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.058540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0885f07-796c-4bda-a095-9f40f821d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.058821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0885f07-796c-4bda-a095-9f40f821d8fb None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:17.059234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:17.060249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:17.061112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:17.061721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0885f07-796c-4bda-a095-9f40f821d8fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.061788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 105/219] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:17.061852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0885f07-796c-4bda-a095-9f40f821d8fb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.061915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.082551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 115/220] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:16 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:17.083977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.084282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.084570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.102642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:17.103890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:17.180834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6be466-28dd-4b45-9359-6aea0f793590 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.183510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 116/221] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:17.184790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.239107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.240145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.240439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.240720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] PATH_INFO: `/v3/users/6f3decc0c34149d582f7422dd47f9046` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.255557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] RBAC: Authorizing `identity:get_user(user_id=6f3decc0c34149d582f7422dd47f9046)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.257075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.258133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 106/222] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/users/6f3decc0c34149d582f7422dd47f9046 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.258914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.276135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.311942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.325918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff258277-b625-4936-9d23-14c41828f9d3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.370435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.371224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.371544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.371916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.372698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:17.372990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.374622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.378194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472483c-6ff8-4e01-b2fd-de24a0269453 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.384624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 117/223] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.385997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.420755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.423103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.423103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.423103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.436625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:17.448655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:17.450260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 107/224] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:17.451051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.512780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f76d6-befc-45bb-91cf-9fb23f9323c3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.552774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.553599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.554018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.554431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.555076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.585208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e6c88-2a61-4c07-bd49-fab8c4530979 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.586099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.625489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.627106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.627904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.628493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] PATH_INFO: `/v3/users/18b2479f09cd4d5ca93b1eaae205daab` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.630562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] RBAC: Authorizing `identity:get_user(user_id=18b2479f09cd4d5ca93b1eaae205daab)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.632120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.632828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.634079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 108/225] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/users/18b2479f09cd4d5ca93b1eaae205daab => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.634661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.635042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.741851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 118/226] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:17.742996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.757282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57847e0c-5643-49b5-bafb-636c80c5e801 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.801885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.802816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.803204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.803845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.805109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.805948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:17.806372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.808987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.821299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 109/227] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.825688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.825903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:17.860995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5535b0-e45b-492c-8ab5-d208e9e4635c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.897227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.898485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.898997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.899473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.900896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.902562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:17.904777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 119/228] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:17.905642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:17.956729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef46a6-963d-4790-b116-dcbce4b57439 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:17.994116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:17.995151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:17.995533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:17.995895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:17.997303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:17.998973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.000892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 110/229] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.001749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.020742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c7441-43f9-46dd-afd0-301b76040eff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.047962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.060417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.061623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.062092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.062487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] PATH_INFO: `/v3/users/18b2479f09cd4d5ca93b1eaae205daab` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.063876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] RBAC: Authorizing `identity:get_user(user_id=18b2479f09cd4d5ca93b1eaae205daab)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.066215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.068167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 120/230] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:17 2022] GET /identity/v3/users/18b2479f09cd4d5ca93b1eaae205daab => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.069202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.086626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.123686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74264159-73a9-43df-9b0c-d9af2b3e904b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.171734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.173116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.173510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.174698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079/users/18b2479f09cd4d5ca93b1eaae205daab/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.177931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8612db90e842afbd07030620e8c079, user_id=18b2479f09cd4d5ca93b1eaae205daab, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.179950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.197092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d7775-90ea-466a-813d-b9d7a2f40c32 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.197988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 111/231] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:18 2022] PUT /identity/v3/projects/4e8612db90e842afbd07030620e8c079/users/18b2479f09cd4d5ca93b1eaae205daab/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:18.200573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.240973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.241647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.241986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.242269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.243328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=6f3decc0c34149d582f7422dd47f9046) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:18.243653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.245216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.251526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 121/232] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/role_assignments?user.id=6f3decc0c34149d582f7422dd47f9046&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.327383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae7cc4-8dc4-44c0-a904-9bfbefb57273 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.360679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.367747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ffafb-df2f-4af9-8aef-ec018ab8312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.368686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ffafb-df2f-4af9-8aef-ec018ab8312f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.369250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ffafb-df2f-4af9-8aef-ec018ab8312f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.369789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ffafb-df2f-4af9-8aef-ec018ab8312f None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.371171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ffafb-df2f-4af9-8aef-ec018ab8312f None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.372782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ffafb-df2f-4af9-8aef-ec018ab8312f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.375215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 112/233] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:18 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:18.381171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:18.482668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:18.538551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27571-d3fa-42d3-872c-7ec371a0b4de None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:18.579278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61af61-1b5c-4c9d-97fd-dc261fc6724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:18.580573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61af61-1b5c-4c9d-97fd-dc261fc6724c None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:18.580916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61af61-1b5c-4c9d-97fd-dc261fc6724c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:18.581203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61af61-1b5c-4c9d-97fd-dc261fc6724c None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079/users/18b2479f09cd4d5ca93b1eaae205daab/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:18.583593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61af61-1b5c-4c9d-97fd-dc261fc6724c None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8612db90e842afbd07030620e8c079, user_id=18b2479f09cd4d5ca93b1eaae205daab, role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:18.585045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61af61-1b5c-4c9d-97fd-dc261fc6724c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:18.603392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 122/234] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:18 2022] PUT /identity/v3/projects/4e8612db90e842afbd07030620e8c079/users/18b2479f09cd4d5ca93b1eaae205daab/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:19.526859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d0ea-4811-42f8-9993-f7db31caf375 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.527371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d0ea-4811-42f8-9993-f7db31caf375 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.527445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d0ea-4811-42f8-9993-f7db31caf375 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.529174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 113/235] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:19.534632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a183dd-bff4-4dae-9a70-c2143966d1f4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.534929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a183dd-bff4-4dae-9a70-c2143966d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.535214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a183dd-bff4-4dae-9a70-c2143966d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.562254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a9a183dd-bff4-4dae-9a70-c2143966d1f4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:19.563461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a9a183dd-bff4-4dae-9a70-c2143966d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:19.660333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 123/236] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:19.700210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d0ea-4811-42f8-9993-f7db31caf375 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:19.762885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fede85-8df4-44e2-8e42-2bbb09fb3faa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.763442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fede85-8df4-44e2-8e42-2bbb09fb3faa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.763948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fede85-8df4-44e2-8e42-2bbb09fb3faa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.768227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 124/237] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:19.773614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.774163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.774666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.800206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:19.801418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:19.866394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d0ea-4811-42f8-9993-f7db31caf375 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:19.878134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:19.909448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 125/238] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:19.925983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:19.926797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:19.927090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:19.927373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:19.950049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:19.959697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:19.960578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:19.961389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:19.962027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:19.962099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 114/239] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:19.966664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:20.062090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:20.109012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb0a69-3d59-4011-ada4-687582f8a645 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:20.140722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc81054-41a9-4dae-9ede-77287c24e922 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:20.155970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:20.156971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.157463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.158614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.159916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:20.160440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:20.162094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:20.181439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 115/240] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:20.187245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:20.190399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:20.191222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.191729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.192190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.198368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:20.200120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:20.202584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 126/241] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:20.208163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:20.288483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:20.300217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b16b-6819-419d-a0b2-52ae193f974f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:20.324607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:20.345095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:20.346380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.346874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.347352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] PATH_INFO: `/v3/users/eda01427ea0c41e3ad01d57ff83fcefb` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.371804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] RBAC: Authorizing `identity:get_user(user_id=eda01427ea0c41e3ad01d57ff83fcefb)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:20.373291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:20.375235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 116/242] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:20 2022] GET /identity/v3/users/eda01427ea0c41e3ad01d57ff83fcefb => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:20.382754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:20.403295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44fbc6-3262-4871-bb26-eb1b1c9e3571 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:20.472079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599013b1-8ee6-4360-bc4d-a92c6da89e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:20.472770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599013b1-8ee6-4360-bc4d-a92c6da89e35 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.473059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599013b1-8ee6-4360-bc4d-a92c6da89e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.473340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599013b1-8ee6-4360-bc4d-a92c6da89e35 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.474107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599013b1-8ee6-4360-bc4d-a92c6da89e35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:20.475829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599013b1-8ee6-4360-bc4d-a92c6da89e35 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:20.512529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 127/243] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:20 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:29:20.575656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce182e50-af8d-4377-9f2e-6687e4e294e3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:20.624601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:20.625327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.625620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.625923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.689143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:20.722064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:20.723629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:20.801236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20478d3-90c5-4745-a032-c9b34c5f2787 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.801908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20478d3-90c5-4745-a032-c9b34c5f2787 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.801908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20478d3-90c5-4745-a032-c9b34c5f2787 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.803432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 128/244] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:20.808977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:20.809244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:20.809532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:20.837312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:20.838843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:20.864841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 117/245] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:20.871480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:20.925765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 129/246] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:20.946412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:20.974010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:20.986086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:20.997500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d37b77-58b4-4d87-92da-efef6d01b580 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.041150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360da15-9f27-4884-9ea1-b62dba94a2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.042179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360da15-9f27-4884-9ea1-b62dba94a2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.043169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360da15-9f27-4884-9ea1-b62dba94a2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.043664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360da15-9f27-4884-9ea1-b62dba94a2a4 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.045405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360da15-9f27-4884-9ea1-b62dba94a2a4 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:21.047413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360da15-9f27-4884-9ea1-b62dba94a2a4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:21.050213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 118/247] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:20 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:21.052301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9e298-940b-4baf-b1fd-2766a55faa4a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.052772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9e298-940b-4baf-b1fd-2766a55faa4a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.053202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9e298-940b-4baf-b1fd-2766a55faa4a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.054836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 119/248] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:21.057170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9e298-940b-4baf-b1fd-2766a55faa4a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.077106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8757c54-2581-4b5e-9bcd-9cc0a6f57fca None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.120562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b5edc-b5ba-4b43-a517-079cc7831545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.121448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5edc-b5ba-4b43-a517-079cc7831545 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.121865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5edc-b5ba-4b43-a517-079cc7831545 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.122132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5edc-b5ba-4b43-a517-079cc7831545 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.139145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b5edc-b5ba-4b43-a517-079cc7831545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-948b5edc-b5ba-4b43-a517-079cc7831545 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:21.148070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:21.148972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:21.149797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 130/249] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:21.155111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.155377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.155607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.175035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:21.176311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:21.197601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9e298-940b-4baf-b1fd-2766a55faa4a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.242729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.243402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.243694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.243980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.244840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] RBAC: Adding query filter params (role.id=27f359bbb37c4303b502646b2b96ca90, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=eda01427ea0c41e3ad01d57ff83fcefb) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:21.245160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:21.246651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:21.256402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 120/250] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/role_assignments?user.id=eda01427ea0c41e3ad01d57ff83fcefb&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=27f359bbb37c4303b502646b2b96ca90 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:21.257613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.274490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 131/251] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:21.287017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:21.352007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.409728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:21.410466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cdf38-ac58-44e4-9346-4d38f50c427f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.454533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.455344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.455640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.455924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.457029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:21.457029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:21.461692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:21.477452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 121/252] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:21.482857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.593718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42b7ee-6c7d-4a5e-ab55-4a2219b78b57 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.622893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:21.657527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.658950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.659356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.663708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61395c3-435d-4db7-9799-2f37ff0edbf5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.666212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.687842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:21.696112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:21.697079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:21.697904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 132/253] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:21.705293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.735797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.737284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.737574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.737909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] PATH_INFO: `/v3/users/6f3decc0c34149d582f7422dd47f9046` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.742435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] RBAC: Authorizing `identity:get_user(user_id=6f3decc0c34149d582f7422dd47f9046)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:21.743954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:21.745722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 122/254] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/users/6f3decc0c34149d582f7422dd47f9046 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:21.751196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.865758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bc8ca-e653-4b8f-a104-620b69683bcb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.880295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:21.909700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02728ff2-8c1c-4ef9-9a04-a97f8ab3743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.909700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02728ff2-8c1c-4ef9-9a04-a97f8ab3743a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.909700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02728ff2-8c1c-4ef9-9a04-a97f8ab3743a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.909700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02728ff2-8c1c-4ef9-9a04-a97f8ab3743a None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.910640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02728ff2-8c1c-4ef9-9a04-a97f8ab3743a None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:21.912121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02728ff2-8c1c-4ef9-9a04-a97f8ab3743a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:21.915774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 123/255] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:21.918024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c80f4-fc9e-4419-8ed4-88e76a41b39d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.919153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c80f4-fc9e-4419-8ed4-88e76a41b39d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.919746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c80f4-fc9e-4419-8ed4-88e76a41b39d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.923132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 124/256] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:21.924648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c80f4-fc9e-4419-8ed4-88e76a41b39d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:21.927716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31916e-7232-4a7c-831a-2b3b3265dac2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:21.959457 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:21.987298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:21.987995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:21.988291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:21.988573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:21.989343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:21.989677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:21.991093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9c01e-31bd-4082-85c2-a24e19d368b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.008313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 133/257] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:22.009614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.009942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.010232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.037407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:22.038831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:22.110916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 134/258] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:22.112359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.166949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c80f4-fc9e-4419-8ed4-88e76a41b39d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.195817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.211642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.212601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.212892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.213341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079/users/6f3decc0c34149d582f7422dd47f9046/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.215927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8612db90e842afbd07030620e8c079, user_id=6f3decc0c34149d582f7422dd47f9046, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.217430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.238813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 125/259] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:21 2022] PUT /identity/v3/projects/4e8612db90e842afbd07030620e8c079/users/6f3decc0c34149d582f7422dd47f9046/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:22.241277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.265198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f3603-dcef-433a-a0ac-375ee81b5c17 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.324107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.325184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.325479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.325788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] PATH_INFO: `/v3/users/18b2479f09cd4d5ca93b1eaae205daab` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.331079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] RBAC: Authorizing `identity:get_user(user_id=18b2479f09cd4d5ca93b1eaae205daab)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.335285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.339916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 135/260] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/users/18b2479f09cd4d5ca93b1eaae205daab => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:22.346431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.405874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.469044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a1b6d-89fc-481d-9281-557be07f0abf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.541416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.542285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.542581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.542870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.565428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae0579-6d42-49b2-890b-12c2226c20d9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.574670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:22.584826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:22.585881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:22.586683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 126/261] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:22.592306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.637079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.637951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.638257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.638547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.656738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.748036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d406-3cc2-4deb-af60-5c567633e80b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:22.787094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:22.794973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.796595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.805592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:22.806321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:22.806618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:22.806911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:22.807748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:22.808118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:22.809500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:22.825583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 127/262] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:22.832296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:22.994798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 136/263] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:23.001416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.039635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0b289-542b-4872-bd55-064fd7872bc8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.092818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.167367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.168638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.169002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.169312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] PATH_INFO: `/v3/users/18b2479f09cd4d5ca93b1eaae205daab` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.170731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] RBAC: Authorizing `identity:get_user(user_id=18b2479f09cd4d5ca93b1eaae205daab)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.172189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.177709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec881d-dce3-4cf4-9864-c9ef00d73b9c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.178547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 128/264] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:22 2022] GET /identity/v3/users/18b2479f09cd4d5ca93b1eaae205daab => generated 283 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:23.186577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.243038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.244223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.244669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.245150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.250456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.252212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.254306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 137/265] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:23.260751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.392019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.418981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2251f-56f9-4904-b58a-1a4c3897acde None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.462833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.488211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.488935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.489260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.489567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.490466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=18b2479f09cd4d5ca93b1eaae205daab) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:23.490808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.492357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791fdef-4523-4bfe-8f1f-93e8ccc0e74f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.505700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 138/266] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/role_assignments?user.id=18b2479f09cd4d5ca93b1eaae205daab&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:23.516911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe64649-d4ec-46ab-8a49-babea057912a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:23.622177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:23.623143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.623599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.624076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.647009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85996e9-477e-4cb2-a5d7-d2f07857654f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.647611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85996e9-477e-4cb2-a5d7-d2f07857654f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.648114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85996e9-477e-4cb2-a5d7-d2f07857654f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.650821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 139/267] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:23.655382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:23.655758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:23.656273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:23.698828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:23.700395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:23.760987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:23.765700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:23.824168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 140/268] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:23.863182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:23.873649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:23.924995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 129/269] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:23.931502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.053637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac16d4-8fde-491b-a84f-7aaee95ae46d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.057101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:24.065949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a48e5-5ac7-4f70-b33d-7898ba575c70 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.116058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.117224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.117647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.118122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.119658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.121428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.123406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.124098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 130/270] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:24.129753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.129753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.129753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.131148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.150717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:24.164122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:24.165267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:24.166321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 141/271] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:24.171514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:24.173163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.273178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03eb5f1-0e88-4b16-97c0-0c4ab0dc899b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.310200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.310832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.311143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.311492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.312371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] RBAC: Adding query filter params (role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=18b2479f09cd4d5ca93b1eaae205daab) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:24.312717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.314400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3a9b-2ead-4837-ba32-07db901d8522 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.320802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 131/272] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/role_assignments?user.id=18b2479f09cd4d5ca93b1eaae205daab&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:24.322001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a53b-087c-4464-9ba8-da0e8fb708fd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.322352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a53b-087c-4464-9ba8-da0e8fb708fd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.322640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a53b-087c-4464-9ba8-da0e8fb708fd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.328045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 132/273] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:24.329651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43d3e-f421-421e-86a8-5564fa5fc992 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.330644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43d3e-f421-421e-86a8-5564fa5fc992 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.330930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43d3e-f421-421e-86a8-5564fa5fc992 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.337110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 133/274] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:24.337110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.337110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.337450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1cbf-5914-4e1a-b87c-bf6b7a2cbe75 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.337530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.355784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:24.355784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:24.394135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.395444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.395842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.396233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.397144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:24.397573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.400471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0986430-ee6b-4707-9c2d-7f31c0cd7407 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.412695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 142/275] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:24.414954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.415368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.415935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.438620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 134/276] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:24.445861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:24.445861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:24.447223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.466179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:24.526029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:24.528848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 143/277] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:24.529527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.602720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b42e20-5f8f-4e23-a2a6-7975dbdc6b72 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.646482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.647365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.647655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.647931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] PATH_INFO: `/v3/users/626a92ced7a44eb1899ff7552f187286` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.665298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] RBAC: Authorizing `identity:get_user(user_id=626a92ced7a44eb1899ff7552f187286)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.666784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.666907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bfc1d-719c-4e76-bca9-e7ebc9abfb18 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.670524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 135/278] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/users/626a92ced7a44eb1899ff7552f187286 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:24.670524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.707437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.708238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.708528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.708811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.710007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.711455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.713214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 144/279] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:24.713853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.739556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:24.763771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:24.786567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057e2d4-5183-4fcc-826e-48e513171c85 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.822963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29736564-629a-45b3-bcbd-dcdc7d9bfbad None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:24.827638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.828699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.829212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.829724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.842192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:24.850574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:24.851482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:24.852280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 136/280] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:24.852983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.862347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:24.863105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:24.863399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:24.863683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:24.864527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, user.id=626a92ced7a44eb1899ff7552f187286, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:24.864846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:24.866331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:24.873718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 145/281] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/role_assignments?user.id=626a92ced7a44eb1899ff7552f187286&scope.system=all&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:24.873718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:24.980834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee73409-72c1-40f4-aacc-ed46e836b6b3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.018641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.023735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.024432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.024727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.025016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.025785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:25.026880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0919c-e67c-447b-bdcc-e3a854a96813 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.027272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:25.038974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.056449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 137/282] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:24 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:25.081280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.082702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.083199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.083660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.084632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:25.085140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:25.086811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:25.098505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 146/283] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:25.104474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.229188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3f7d4-a7f5-4187-8aeb-660dfdc30959 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.268166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d1c00-b47b-40ee-82d8-d73f71a239eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.269402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d1c00-b47b-40ee-82d8-d73f71a239eb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.270128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d1c00-b47b-40ee-82d8-d73f71a239eb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.270817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d1c00-b47b-40ee-82d8-d73f71a239eb None admin] PATH_INFO: `/v3/users/eda01427ea0c41e3ad01d57ff83fcefb` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.273096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d1c00-b47b-40ee-82d8-d73f71a239eb None admin] RBAC: Authorizing `identity:get_user(user_id=eda01427ea0c41e3ad01d57ff83fcefb)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:25.274983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d1c00-b47b-40ee-82d8-d73f71a239eb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:25.277144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 138/284] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:25 2022] GET /identity/v3/users/eda01427ea0c41e3ad01d57ff83fcefb => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:25.282942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.400600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a228b-18ca-4703-b6d5-5eb8880e7106 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.436844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1569ef0-0aa2-48c9-b958-c5df452ae3c6 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.437151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1569ef0-0aa2-48c9-b958-c5df452ae3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.437441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1569ef0-0aa2-48c9-b958-c5df452ae3c6 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.438737 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.438883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 139/285] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:25.440220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.440545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.440851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.442296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:25.446424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:25.446424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 147/286] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:25 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:25.448546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.448840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.449126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.456839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.477400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:25.479048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:25.480384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.570444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.574181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb28c-46bd-47e2-9270-4b13770965aa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.583105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 140/287] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:25.618632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.634708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.635620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.635915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.636196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079/users/eda01427ea0c41e3ad01d57ff83fcefb/roles/27f359bbb37c4303b502646b2b96ca90` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.639584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8612db90e842afbd07030620e8c079, user_id=eda01427ea0c41e3ad01d57ff83fcefb, role_id=27f359bbb37c4303b502646b2b96ca90)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:25.641047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:25.656196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 148/288] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:29:25 2022] PUT /identity/v3/projects/4e8612db90e842afbd07030620e8c079/users/eda01427ea0c41e3ad01d57ff83fcefb/roles/27f359bbb37c4303b502646b2b96ca90 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:25.743672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678e7ae-f50d-47af-9e76-5e6368a9e593 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:25.783396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:25.784223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:25.784513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:25.784797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:25.797549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:25.806929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:25.807876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:25.808666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:25.809256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 141/289] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:25.815046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:25.945397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:25.975627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddacd8-80a4-44b1-ba2a-54c8163f810c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.011877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.012566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.012850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.013133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.013924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:26.014240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.015631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.024723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 149/290] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.030199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.107331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.166386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5917da-fcf1-4000-8a33-709cbff179d4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.219388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.220211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.221230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.221588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] PATH_INFO: `/v3/users/6f3decc0c34149d582f7422dd47f9046` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.222874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] RBAC: Authorizing `identity:get_user(user_id=6f3decc0c34149d582f7422dd47f9046)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.224350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.226028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 142/291] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/users/6f3decc0c34149d582f7422dd47f9046 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.231498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.345149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7d259-2e19-4e3b-8e43-8ecc058e0838 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.381462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.382166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.382458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.382741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.391544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.463513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.465191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.567166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 150/292] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:26.572991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.715379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aa454-019d-4a42-bc08-4969fb0ff89f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.750087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.771262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afac09-1221-476f-a298-3299a05224f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.772267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afac09-1221-476f-a298-3299a05224f8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.772581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afac09-1221-476f-a298-3299a05224f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.772927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afac09-1221-476f-a298-3299a05224f8 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.778427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afac09-1221-476f-a298-3299a05224f8 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.779899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afac09-1221-476f-a298-3299a05224f8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.781811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 143/293] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:26.790757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:26.862916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:26.922271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0758c-1409-4f4b-8028-fedc8ca28255 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:26.975097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:26.975807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:26.976119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:26.976405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:26.977329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=6f3decc0c34149d582f7422dd47f9046) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:26.977651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:26.979140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423b8f0-aef0-4c8f-ad7c-0a104c33b831 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:26.991065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 151/294] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:26 2022] GET /identity/v3/role_assignments?user.id=6f3decc0c34149d582f7422dd47f9046&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.261602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da659f-8427-4c36-8f10-e7fedf9d698f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.262106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da659f-8427-4c36-8f10-e7fedf9d698f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.262179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da659f-8427-4c36-8f10-e7fedf9d698f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.263630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 144/295] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:27.271181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.272023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.272503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.296428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:27.297974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:27.397197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 152/296] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:27.421639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da659f-8427-4c36-8f10-e7fedf9d698f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.556386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da659f-8427-4c36-8f10-e7fedf9d698f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.597712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.598900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.599388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.599864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.610961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:27.619312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:27.620172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:27.620976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:27.625724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 145/297] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:27.632423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.663918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.737518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c78188-e231-4f56-83db-e8c593e32626 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.776912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.777596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.777920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.778199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.778963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:27.779275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.780652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.790612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 153/298] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:27.797347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.855591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:27.913546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d075f46-88c3-49db-b14f-613c381ec571 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:27.957610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036f33c-e37e-4365-9de8-8f3f316a6832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:27.958865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036f33c-e37e-4365-9de8-8f3f316a6832 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.959341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036f33c-e37e-4365-9de8-8f3f316a6832 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.959826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036f33c-e37e-4365-9de8-8f3f316a6832 None admin] PATH_INFO: `/v3/users/626a92ced7a44eb1899ff7552f187286` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:27.961239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036f33c-e37e-4365-9de8-8f3f316a6832 None admin] RBAC: Authorizing `identity:get_user(user_id=626a92ced7a44eb1899ff7552f187286)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:27.962910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036f33c-e37e-4365-9de8-8f3f316a6832 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:27.965241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 146/299] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:27 2022] GET /identity/v3/users/626a92ced7a44eb1899ff7552f187286 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:27.972263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:27.997455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7de189-c4fd-47df-ac41-05ce43bc41e1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:27.997784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7de189-c4fd-47df-ac41-05ce43bc41e1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:27.998056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7de189-c4fd-47df-ac41-05ce43bc41e1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.000129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 147/300] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:28.005416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dde433-186c-4a5f-828b-be54fab6fce6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.005734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dde433-186c-4a5f-828b-be54fab6fce6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.006024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dde433-186c-4a5f-828b-be54fab6fce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.024381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-47dde433-186c-4a5f-828b-be54fab6fce6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:28.025609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-47dde433-186c-4a5f-828b-be54fab6fce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:28.098874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.106298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059994-b45f-457e-b678-4a51fd670e71 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.115632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 148/301] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:28.117153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad7ea-9159-4a87-b76e-6b1c436b3d6a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.117451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad7ea-9159-4a87-b76e-6b1c436b3d6a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.117765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad7ea-9159-4a87-b76e-6b1c436b3d6a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.119154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 149/302] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:28.124225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.124525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.124815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.143989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:28.145188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:28.152926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.154018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.154480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.154910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] PATH_INFO: `/v3/system/users/626a92ced7a44eb1899ff7552f187286/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.156555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=626a92ced7a44eb1899ff7552f187286, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.158265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.167744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 154/303] 10.222.0.57 () {58 vars in 1334 bytes} [Wed Jun 15 14:29:27 2022] PUT /identity/v3/system/users/626a92ced7a44eb1899ff7552f187286/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:28.168830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.197497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.223944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 150/304] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:28.249961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.308770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acaf55-a74f-412a-bdcb-bdfd6789429f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.353627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.355673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.356038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.356446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.357969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.359642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.362486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 155/305] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:28.370991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.400407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c279d-3950-40de-a9c5-c118636f40b2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.424696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.443935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7684b3-f0e9-43b4-94f4-846a2e223368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.444754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7684b3-f0e9-43b4-94f4-846a2e223368 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.445039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7684b3-f0e9-43b4-94f4-846a2e223368 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.445315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7684b3-f0e9-43b4-94f4-846a2e223368 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.460482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7684b3-f0e9-43b4-94f4-846a2e223368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cf7684b3-f0e9-43b4-94f4-846a2e223368 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:28.468376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.469329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:28.470724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 151/306] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:28.471922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6234603-45df-4d4d-9ac8-16772cb1aa0a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.471922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6234603-45df-4d4d-9ac8-16772cb1aa0a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.472213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6234603-45df-4d4d-9ac8-16772cb1aa0a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.477273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 152/307] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:28.477273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6234603-45df-4d4d-9ac8-16772cb1aa0a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.505466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b710116-86cc-4054-9d14-b3aab0810601 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.523941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.545151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103b421-66c6-4867-9cdc-88b2d0b7e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.545864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103b421-66c6-4867-9cdc-88b2d0b7e641 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.546158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103b421-66c6-4867-9cdc-88b2d0b7e641 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.546451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103b421-66c6-4867-9cdc-88b2d0b7e641 None admin] PATH_INFO: `/v3/groups` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.547268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103b421-66c6-4867-9cdc-88b2d0b7e641 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.549467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103b421-66c6-4867-9cdc-88b2d0b7e641 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.567789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 156/308] 10.222.0.57 () {60 vars in 1134 bytes} [Wed Jun 15 14:29:28 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:28.569955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.570430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.570874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.599265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:28.600427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6234603-45df-4d4d-9ac8-16772cb1aa0a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.601092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:28.650117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.650728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.651016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.651299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.652068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:28.652390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.653822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.669077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 153/309] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:28.671443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.714167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 157/310] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:28.739029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.757676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.807903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b8cea-3d0a-44df-9ed3-238a5afde596 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.812757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:28.855248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.856689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.857198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.857633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] PATH_INFO: `/v3/users/8296401155134bf9af59c59109eaa0d5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.871166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef93626-0c54-45ec-9c23-bb04df4a5930 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:28.873411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] RBAC: Authorizing `identity:get_user(user_id=8296401155134bf9af59c59109eaa0d5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:28.875165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:28.877858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 154/311] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/users/8296401155134bf9af59c59109eaa0d5 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:28.884471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:28.916627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:28.918470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:28.919081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:28.919646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:28.935976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:28.947801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:28.948711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:28.949512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:28.955933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 158/312] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:28.956954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.021262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6e833-7bde-41a7-ac8c-b7f6e331ab27 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.040166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.064686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.065342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.065630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.065936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.066766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] RBAC: Adding query filter params (role.id=27f359bbb37c4303b502646b2b96ca90, user.id=8296401155134bf9af59c59109eaa0d5, scope.system=all) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.067086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.068512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.073798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 155/313] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/role_assignments?user.id=8296401155134bf9af59c59109eaa0d5&scope.system=all&role.id=27f359bbb37c4303b502646b2b96ca90 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.074312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35ee5-546e-4103-ab7d-9ac4be97eb51 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.114855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.115923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.116406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.116893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.117882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.118406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.120001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.143724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 159/314] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.149923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.187356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.270408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03c27-8007-4e9e-86a4-f773061e363e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.307661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.309042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.309529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.310041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] PATH_INFO: `/v3/users/eda01427ea0c41e3ad01d57ff83fcefb` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.311617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] RBAC: Authorizing `identity:get_user(user_id=eda01427ea0c41e3ad01d57ff83fcefb)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.313350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.316384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 156/315] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/users/eda01427ea0c41e3ad01d57ff83fcefb => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.322207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.434354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745a3c-3167-4fc7-b8d2-2bb534533103 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.482339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.483381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.483860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.484328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.494364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.560420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.562302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.673978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 160/316] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:29.676114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.775638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.787607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212814ca-daa5-4066-9bca-7a08512314fe None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.824385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336934e7-d299-473e-97ea-34356710ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.825589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336934e7-d299-473e-97ea-34356710ece2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.826088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336934e7-d299-473e-97ea-34356710ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.826810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336934e7-d299-473e-97ea-34356710ece2 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.828185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336934e7-d299-473e-97ea-34356710ece2 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.831075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336934e7-d299-473e-97ea-34356710ece2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:29.833289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 157/317] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:29.839438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:29.886133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:29.952442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e2997-5e00-4a42-95b4-05ac4c81d6c9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:29.989055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:29.990164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:29.990644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:29.991111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:29.992134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] RBAC: Adding query filter params (role.id=27f359bbb37c4303b502646b2b96ca90, scope.project.id=4e8612db90e842afbd07030620e8c079, user.id=eda01427ea0c41e3ad01d57ff83fcefb) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:29.992661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:29.994343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43615ae-782a-406f-9e1d-cb5117fb1632 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.000866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 161/318] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:29 2022] GET /identity/v3/role_assignments?user.id=eda01427ea0c41e3ad01d57ff83fcefb&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=27f359bbb37c4303b502646b2b96ca90 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.372664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31b0f-7c92-4181-8ae4-da0a6ae21e8c None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.373036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31b0f-7c92-4181-8ae4-da0a6ae21e8c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.373236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31b0f-7c92-4181-8ae4-da0a6ae21e8c None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.374804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 158/319] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:30.380330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.380764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.380922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.397783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:30.398977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:30.473705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 162/320] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:30.497143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31b0f-7c92-4181-8ae4-da0a6ae21e8c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.602304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31b0f-7c92-4181-8ae4-da0a6ae21e8c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.638532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3285b24-d349-4618-8062-aa1d5ef845ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.639268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3285b24-d349-4618-8062-aa1d5ef845ac None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.639559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3285b24-d349-4618-8062-aa1d5ef845ac None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.639842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3285b24-d349-4618-8062-aa1d5ef845ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.650783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3285b24-d349-4618-8062-aa1d5ef845ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e3285b24-d349-4618-8062-aa1d5ef845ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:30.659183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:30.660088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:30.660880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:30.662211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 159/321] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:30.666122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.698176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.774243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbc28b-eff1-4ba4-933a-3013ad64168b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.811509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.812184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.812475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.812753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.813512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:30.813851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.815263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.818771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649421c1-3f15-4c5d-b92f-5501cfe3b22a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.819216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649421c1-3f15-4c5d-b92f-5501cfe3b22a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.819400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649421c1-3f15-4c5d-b92f-5501cfe3b22a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.820718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 160/322] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:30.823834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 163/323] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.827212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.827503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.827777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.830084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.844393 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:30.846169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:30.859552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.918631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 161/324] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:30.937914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:30.942161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:30.947701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e17876-c4f0-4334-8e6e-74d853ce7005 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:30.981891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:30.982784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:30.983076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:30.983360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] PATH_INFO: `/v3/users/626a92ced7a44eb1899ff7552f187286` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:30.984575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] RBAC: Authorizing `identity:get_user(user_id=626a92ced7a44eb1899ff7552f187286)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:30.986548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:30.988365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 164/325] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/users/626a92ced7a44eb1899ff7552f187286 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:30.993813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.049021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93be58-1740-48ca-be19-d111f15390a5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.084668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.085459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.085771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.086052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.087294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.089263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.091063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 162/326] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:31.096293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.097473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7223e37-01a1-4fff-8d7b-1cc9b52d24ca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.114553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:31.134850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.135621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.135984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.136342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.137293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] RBAC: Adding query filter params (role.id=bdd97d4caedc488bac31aec6d879f431, user.id=626a92ced7a44eb1899ff7552f187286, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:31.137704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.139255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059bcaf-115f-44e3-8580-450e639a4cb6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.152164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 165/327] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:30 2022] GET /identity/v3/role_assignments?user.id=626a92ced7a44eb1899ff7552f187286&scope.system=all&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:31.211762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a2435-547a-48d6-adc7-dda3550ad900 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.249747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.250447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.250736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.251016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] PATH_INFO: `/v3/groups` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.252497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781a18-6ad4-4100-9dfd-782744a6cda6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.252627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.252843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781a18-6ad4-4100-9dfd-782744a6cda6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.253130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781a18-6ad4-4100-9dfd-782744a6cda6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.254200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.254750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 166/328] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:31.260419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.260997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.261284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.275800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 163/329] 10.222.0.57 () {60 vars in 1134 bytes} [Wed Jun 15 14:29:31 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:29:31.289020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:31.290309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:31.309035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:31.377537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 167/330] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:31.400173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.483298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:31.508726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c56fb1-45d9-4507-aab4-8b093586c1ac None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.544185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.544988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.545274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.545556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.556597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:31.565431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:31.566413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:31.567192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 164/331] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:31.572764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.684949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87c6ab-a152-4868-b7fb-32fd2688274b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.722559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.723484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.723918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.724370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.725273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:31.725764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.727376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.736346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 168/332] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:31.742066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.848965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce21eb6-6260-43f8-bbc9-b388b36aed22 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:31.885847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c68be3-5386-4591-b62f-9af93a7b7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:31.886973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c68be3-5386-4591-b62f-9af93a7b7118 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:31.887404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c68be3-5386-4591-b62f-9af93a7b7118 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:31.887849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c68be3-5386-4591-b62f-9af93a7b7118 None admin] PATH_INFO: `/v3/users/8296401155134bf9af59c59109eaa0d5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:31.889205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c68be3-5386-4591-b62f-9af93a7b7118 None admin] RBAC: Authorizing `identity:get_user(user_id=8296401155134bf9af59c59109eaa0d5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:31.890827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c68be3-5386-4591-b62f-9af93a7b7118 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:31.893568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 165/333] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:31 2022] GET /identity/v3/users/8296401155134bf9af59c59109eaa0d5 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:31.901689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:31.917175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:32.016662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541c410-7aea-41bf-bae8-da8b8a5e8b68 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:32.054169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7eec8-57c8-4d6b-a549-eb14375101c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:32.055053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7eec8-57c8-4d6b-a549-eb14375101c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:32.055348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7eec8-57c8-4d6b-a549-eb14375101c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:32.055634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7eec8-57c8-4d6b-a549-eb14375101c6 None admin] PATH_INFO: `/v3/system/users/8296401155134bf9af59c59109eaa0d5/roles/27f359bbb37c4303b502646b2b96ca90` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:32.057276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7eec8-57c8-4d6b-a549-eb14375101c6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8296401155134bf9af59c59109eaa0d5, role_id=27f359bbb37c4303b502646b2b96ca90)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:32.058691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7eec8-57c8-4d6b-a549-eb14375101c6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:32.067710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 169/334] 10.222.0.57 () {58 vars in 1334 bytes} [Wed Jun 15 14:29:31 2022] PUT /identity/v3/system/users/8296401155134bf9af59c59109eaa0d5/roles/27f359bbb37c4303b502646b2b96ca90 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:33.975381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c20b2-866a-4a38-a5eb-68f1fa91b08f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.975920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c20b2-866a-4a38-a5eb-68f1fa91b08f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.975996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c20b2-866a-4a38-a5eb-68f1fa91b08f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:33.977561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 166/335] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:33.984212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:33.984671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:33.984865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.010707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:34.011967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:34.020718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bad70-75d9-4781-ba69-38e9c21244dd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.021016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bad70-75d9-4781-ba69-38e9c21244dd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.021291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bad70-75d9-4781-ba69-38e9c21244dd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.022705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 167/336] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:34.029943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.030243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.030527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.040889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.061215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:34.062463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:34.171859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 170/337] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:34.215707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.216145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 168/338] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:34.228425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.272887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.396978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01af775-795c-4e05-9d90-218c3d53b50a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.458380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81292905-124f-40d0-849e-5fab10fc6a13 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.464214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e56844-6541-4230-a0bf-b9a5bc9cc513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.465373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e56844-6541-4230-a0bf-b9a5bc9cc513 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.465905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e56844-6541-4230-a0bf-b9a5bc9cc513 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.466386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e56844-6541-4230-a0bf-b9a5bc9cc513 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.486696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e56844-6541-4230-a0bf-b9a5bc9cc513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-20e56844-6541-4230-a0bf-b9a5bc9cc513 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:34.495155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.496037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:34.496818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:34.501359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 171/339] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:34.502634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a307f0c-fd00-4b3d-a4b4-6973d4e7f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.503182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212c72e-f413-4c61-850e-160b50bb0dad None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.503461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307f0c-fd00-4b3d-a4b4-6973d4e7f440 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.503749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307f0c-fd00-4b3d-a4b4-6973d4e7f440 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.504029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307f0c-fd00-4b3d-a4b4-6973d4e7f440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.504350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212c72e-f413-4c61-850e-160b50bb0dad None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.504818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212c72e-f413-4c61-850e-160b50bb0dad None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.506643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 172/340] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:34.507536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212c72e-f413-4c61-850e-160b50bb0dad None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.507997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.521368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a307f0c-fd00-4b3d-a4b4-6973d4e7f440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0a307f0c-fd00-4b3d-a4b4-6973d4e7f440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:34.531801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:34.533257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:34.534152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:34.544467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 169/341] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:34.544467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.544467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.544467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.575562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:34.577096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:34.601596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.643988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212c72e-f413-4c61-850e-160b50bb0dad None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.692468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.693520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.694208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.694685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.695649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.696173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.697792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.699133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 170/342] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:34.700118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.709078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 173/343] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.718915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.841421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b8e97-3565-44fd-ab8f-469f8dda26f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.856350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160b8a6-a5a8-4ba0-8c78-d6ab459ad459 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:34.872716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.882348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.883137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:34.884148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.884634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.885131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.906895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:34.907641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:34.908557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:34.908843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:34.909615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:34.909955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.911455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:34.925180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 171/344] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:34.926559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:34.964985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:34.966780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.053822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734770-75c9-411b-8147-dfb3264763fd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.073049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 174/345] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:35.074583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.094629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.095783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.096219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.096643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.104080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.110906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:35.120812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:35.121986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:35.122898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:35.125095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 172/346] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:35.126062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.182519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5994789-9644-4432-a49a-ef92f80e6f21 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.183687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.219608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.220927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.221441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.221764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.248664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fc194-b07c-4244-8ecf-eebf736e0adf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.286981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.288381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.288911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.289337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.290747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.292438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.294734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 173/347] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.295093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.295916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.296650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.325692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.417761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 175/348] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:35.418631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.435775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6468355-af3a-4b3b-9032-18fa8b914a00 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.457244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.488220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.488905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.489200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.489486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.490555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.491256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.492659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.504921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 174/349] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.506117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.558363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47705cf-4eb8-4f9e-a486-3c49b72b6219 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.597856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.611660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea30c6-00ea-411f-a640-4e20d835e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.612539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea30c6-00ea-411f-a640-4e20d835e35a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.612826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea30c6-00ea-411f-a640-4e20d835e35a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.613104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea30c6-00ea-411f-a640-4e20d835e35a None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.624942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea30c6-00ea-411f-a640-4e20d835e35a None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.626884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea30c6-00ea-411f-a640-4e20d835e35a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.628943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 176/350] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.631061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc1bc-b2d7-4cad-8db5-d5f40e7f8138 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.631573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc1bc-b2d7-4cad-8db5-d5f40e7f8138 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.632049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc1bc-b2d7-4cad-8db5-d5f40e7f8138 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.633015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4df37-f13f-4f78-95bf-90123b5d6cbc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.633843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 177/351] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:35.634812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc1bc-b2d7-4cad-8db5-d5f40e7f8138 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.679297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.681032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.681585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.682106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.683652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.686302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.688510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 175/352] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.690191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:35.756872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.773087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc1bc-b2d7-4cad-8db5-d5f40e7f8138 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.815445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237546f-4ec6-41b7-81aa-22de817234ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.816487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237546f-4ec6-41b7-81aa-22de817234ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.817016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237546f-4ec6-41b7-81aa-22de817234ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.817507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237546f-4ec6-41b7-81aa-22de817234ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.824327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b8db0-3126-4cd2-b7fd-ca7ed03f5877 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:35.864873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:35.865192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.865477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.865798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.866679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=3b7e424d403346c5bbf92a367142a6df) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:35.867055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.868610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dea508-6bdb-448d-83f6-2c6aaf291ee5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.876749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 176/353] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:35.877095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:35.877379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:35.877692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:35.895721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:35.897313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:35.899030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:35.908637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237546f-4ec6-41b7-81aa-22de817234ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:35.910365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237546f-4ec6-41b7-81aa-22de817234ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:35.996351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 177/354] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:35.997767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.059640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 178/355] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:36.061070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84b5f-989e-4346-aeb2-33d1bf991ad5 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.061365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84b5f-989e-4346-aeb2-33d1bf991ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.061648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84b5f-989e-4346-aeb2-33d1bf991ad5 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.063054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 179/356] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:36.063886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84b5f-989e-4346-aeb2-33d1bf991ad5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.068559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.150544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fee289-037a-415c-9b31-ed11f7ccbe7a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.179550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.192830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84b5f-989e-4346-aeb2-33d1bf991ad5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.195199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.196086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.196378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.196663 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.197949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.199552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.201685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 178/357] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:35 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.208559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.247565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d25249-96a7-41a5-98a1-e8b71239571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.248315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d25249-96a7-41a5-98a1-e8b71239571a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.248604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d25249-96a7-41a5-98a1-e8b71239571a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.248885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d25249-96a7-41a5-98a1-e8b71239571a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.263555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d25249-96a7-41a5-98a1-e8b71239571a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-54d25249-96a7-41a5-98a1-e8b71239571a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:36.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.275785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:36.276589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 180/358] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.277807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8887a3-3c5c-4c1b-afa8-596fd108ae1c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.277807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8887a3-3c5c-4c1b-afa8-596fd108ae1c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.278134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8887a3-3c5c-4c1b-afa8-596fd108ae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.299804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1c8887a3-3c5c-4c1b-afa8-596fd108ae1c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:36.301041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1c8887a3-3c5c-4c1b-afa8-596fd108ae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:36.336150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c1944-8225-46a4-b48c-2f550ded7ba5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.344433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.382644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.383964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.384258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.384543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.384713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 181/359] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:36.388207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f939a-fb95-47ce-9fd1-f8a037c64631 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.388207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f939a-fb95-47ce-9fd1-f8a037c64631 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.388207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f939a-fb95-47ce-9fd1-f8a037c64631 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.388439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.388528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 182/360] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:36.388598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.388802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f939a-fb95-47ce-9fd1-f8a037c64631 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.391646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 179/361] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.391646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.504622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.530191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f939a-fb95-47ce-9fd1-f8a037c64631 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.530388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa8008-0dc4-4c07-adaa-896bbcf8254c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.571739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.572423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.572709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.572993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.574386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:36.574386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.575625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.586179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 180/362] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.587226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.587526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.588215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.588502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.588781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.589637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=3b7e424d403346c5bbf92a367142a6df) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:36.590134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.591919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf75b1-7e9f-4e1a-9e9a-9699f937649d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.599931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 183/363] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.601045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.601339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.601635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.621156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:36.622599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:36.642048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.719882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 184/364] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:36.721705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.727606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c9dee-857b-4394-952b-813b99d140c8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.792859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.793793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.794086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.794378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.797755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:36.799892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:36.801631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 181/365] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:36.802467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.851878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.882611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de1c10-ed36-4502-b005-ba1b9bfa9bf8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.916853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06778c54-2508-41db-9363-a959c3b5dc3b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:36.919612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:36.925759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.926799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.927231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.927732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:36.939748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:36.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:36.948818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:29:36.949635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:36.952432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 185/366] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:36.953503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:36.956194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:36.956921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:36.957215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:36.957552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.031277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.033362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.065098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff301fb5-03d9-4133-a2d8-016836b0eb36 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.102531 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.103724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.104226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.104793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.108246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.119941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:37.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:37.129958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:37.130731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:37.137877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 186/367] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:37.139027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.143767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 182/368] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:37.144641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.148513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.251890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48423d70-8767-469b-a1b9-dc622b5dbeed None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.256138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346de38-5663-41f6-8687-46067c135f8b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.289296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.290384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.290957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.291449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.292480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=4e8612db90e842afbd07030620e8c079) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.293182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.294297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.294926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.295218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.295507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.296269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.296575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.296680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.297990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.303095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 187/369] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.304306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.307654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 183/370] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.308685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.393775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.403333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.438087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8459b6-a94b-4d7e-a1db-d47923fba63c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.474327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.475212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.475504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.475790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.477270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.478800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.481064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedc4ba-c6eb-4e40-bae2-4048c40c87c2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.483384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 184/371] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.484257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.547335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.548107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.548471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.549129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.550071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.553186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.554848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.573780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 188/372] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.575004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.608184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae55df-48e0-4201-ab1b-7cb51a99efaf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.649949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.651123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.651564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.652064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] PATH_INFO: `/v3/users/8296401155134bf9af59c59109eaa0d5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.654312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] RBAC: Authorizing `identity:get_user(user_id=8296401155134bf9af59c59109eaa0d5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.656175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.658158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 185/373] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/users/8296401155134bf9af59c59109eaa0d5 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.659167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.686055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.722210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006726c8-ad9b-4f39-9e19-630f9ab98cc4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.769237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.783850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf35c2-920e-4c8e-9f49-5c6f6021c24b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.789058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.789976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.790261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.790539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.791737 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.793181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.794827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 189/374] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:37.801093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:37.825377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.827047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.828068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.828800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.928671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.930841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:37.945066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b44974-2a47-4e3f-bb75-1c67e078dcfd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:37.946012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:37.985607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:37.986920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:37.987215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:37.987501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:37.992227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] RBAC: Adding query filter params (role.id=27f359bbb37c4303b502646b2b96ca90, user.id=8296401155134bf9af59c59109eaa0d5, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:37.992227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:37.992227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.009857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 190/375] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/role_assignments?user.id=8296401155134bf9af59c59109eaa0d5&scope.system=all&role.id=27f359bbb37c4303b502646b2b96ca90 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:38.009857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.075809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.087477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 186/376] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:38.093173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.164826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0a62b-2ecc-4f24-a15d-1335e965bf30 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.269281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.272009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.272009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.272009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.272009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=4e8612db90e842afbd07030620e8c079) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:38.272009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.273014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.278111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98b8ab-f491-4370-9b2a-d0a5a0376c98 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.280574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.288104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 191/377] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:38.328306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.330012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.330532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.331027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] PATH_INFO: `/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.332886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] RBAC: Authorizing `identity:get_project(project_id=10e451bce05f4f3c975f7e8a5cfb1440)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.334646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.337128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 187/378] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440 => generated 344 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:38.342468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.458860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8dee-7760-406e-916b-163e7c14edaa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.475733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.500842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c999c-f7a0-4919-ae53-92dc7305ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.501793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c999c-f7a0-4919-ae53-92dc7305ab6d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.502088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c999c-f7a0-4919-ae53-92dc7305ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.502377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c999c-f7a0-4919-ae53-92dc7305ab6d None admin] PATH_INFO: `/v3/groups/acfc9696337540dcb078be5560864544` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.508118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c999c-f7a0-4919-ae53-92dc7305ab6d None admin] RBAC: Authorizing `identity:get_group(group_id=acfc9696337540dcb078be5560864544)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.509604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c999c-f7a0-4919-ae53-92dc7305ab6d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.511556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 192/379] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/groups/acfc9696337540dcb078be5560864544 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:38.518502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.542148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133c94b-6285-4c77-a697-7d57a31f7532 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.542480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133c94b-6285-4c77-a697-7d57a31f7532 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.543061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133c94b-6285-4c77-a697-7d57a31f7532 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.544527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 193/380] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:38.554337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.554628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.554915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.576131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:38.577771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:38.653672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968063a-e88c-40bd-82b9-a146e41f50fa None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.660272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 194/381] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:38.687179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.692940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.693961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.694436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.694896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.695962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] RBAC: Adding query filter params (group.id=acfc9696337540dcb078be5560864544, role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=10e451bce05f4f3c975f7e8a5cfb1440) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:38.696475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:38.698187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:38.706764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 188/382] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/role_assignments?group.id=acfc9696337540dcb078be5560864544&scope.project.id=10e451bce05f4f3c975f7e8a5cfb1440&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:38.846399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38461edb-270d-499f-9eab-cecc8e7cfdb1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:38.859006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:38.888785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:38.889602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:38.889922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:38.890206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:38.904671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:38.912954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:38.913955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:38.914763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:38.916937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 195/383] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:38.922700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:38.981921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.062815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310ce7-ec90-4a91-b095-d2e1a7e9b736 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.102713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.103317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.103606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.103891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.104664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:39.104974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.106394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.121138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.121993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 189/384] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:39.252709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46afd5-204a-4734-a450-c49e28d2b3ce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.292787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.293650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.293961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.294245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.295481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.296941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.299459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 196/385] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:39.304582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.432599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.439347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618863a9-16aa-45c5-a907-c38bc347267d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.481224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17535712-e682-4a7f-b1d8-a009d9d155fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.482134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17535712-e682-4a7f-b1d8-a009d9d155fc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.482424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17535712-e682-4a7f-b1d8-a009d9d155fc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.482705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17535712-e682-4a7f-b1d8-a009d9d155fc None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.484428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17535712-e682-4a7f-b1d8-a009d9d155fc None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.486100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17535712-e682-4a7f-b1d8-a009d9d155fc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.487853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 190/386] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:39 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:39.493575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:39.538308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:39.613149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a65a-b6ce-474b-a735-f7a1f4879393 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:39.650693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a91dc3-8473-4ab1-adba-89f9a8e9cdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:39.651594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91dc3-8473-4ab1-adba-89f9a8e9cdb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:39.651889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91dc3-8473-4ab1-adba-89f9a8e9cdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:39.652173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91dc3-8473-4ab1-adba-89f9a8e9cdb3 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df/groups/3275ae7a3c9b419ca5af63b112f74333/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:39.654225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a91dc3-8473-4ab1-adba-89f9a8e9cdb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7e424d403346c5bbf92a367142a6df, group_id=3275ae7a3c9b419ca5af63b112f74333, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:39.655685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a91dc3-8473-4ab1-adba-89f9a8e9cdb3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:39.673469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 197/387] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:39 2022] PUT /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df/groups/3275ae7a3c9b419ca5af63b112f74333/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:40.142659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0505-4939-4b82-80a4-634eb3cacd89 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.143113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0505-4939-4b82-80a4-634eb3cacd89 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.143409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0505-4939-4b82-80a4-634eb3cacd89 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.145056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 191/388] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:40.150373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.150674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.150958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.171490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:40.172737 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:40.243303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63b0d-f1bb-4fed-af76-a8f8d5df0607 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.243630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63b0d-f1bb-4fed-af76-a8f8d5df0607 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.243881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63b0d-f1bb-4fed-af76-a8f8d5df0607 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.245484 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 192/389] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:40.250794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.251115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.251764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.252931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 198/390] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:40.269839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:40.271021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:40.294565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.350039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.393288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 193/391] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:40.432896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.477280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949c57-52ea-4edb-8871-78091fa3daaa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.531946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.533852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.534321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.534902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.547274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:40.556993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:40.558011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:40.558792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:40.561604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 199/392] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:40.567856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.590723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.605394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096b83c-6744-498b-ac65-995970b6e754 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.675130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.675881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.676166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.676444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.697568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:40.709594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:40.710748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:40.711731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:40.714294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 194/393] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:40.720055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.762025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:40.791986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b40a-ca85-4e50-819c-de0d3d21811a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.861305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.862284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.862678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.863084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.864146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:40.864467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.866003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.887189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 200/394] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:40.890522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d985dd-b9aa-4ee7-aacb-b7161c1c4b0e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:40.892562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.928815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:40.929769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:40.930205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:40.930636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:40.931590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:40.932037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:40.933653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:40.945707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 195/395] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:40.951919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:40.988459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.102389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04256-eca4-45db-8a66-a09ef01b72f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.124495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bee46a-8b31-4dea-a0ee-c992ee1eb801 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.168299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.169179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.169461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.169772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.170971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.172596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.175010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 196/396] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.181174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.194368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619de076-ca91-4372-88ef-da35b59d75ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.195260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619de076-ca91-4372-88ef-da35b59d75ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.195550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619de076-ca91-4372-88ef-da35b59d75ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.195834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619de076-ca91-4372-88ef-da35b59d75ce None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.198818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619de076-ca91-4372-88ef-da35b59d75ce None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.202640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619de076-ca91-4372-88ef-da35b59d75ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.205828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 201/397] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:40 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:41.207463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eae459-de82-4e0d-aee9-a633c1f794ad None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.208057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eae459-de82-4e0d-aee9-a633c1f794ad None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.208569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eae459-de82-4e0d-aee9-a633c1f794ad None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.210132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 202/398] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:41.211125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eae459-de82-4e0d-aee9-a633c1f794ad None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.235142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.286024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.303246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe29d8f-7e50-4c04-80dc-2705a307451d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.339506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb382077-7873-4da0-9f8b-a7f970c13de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.339703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb382077-7873-4da0-9f8b-a7f970c13de2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.339933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb382077-7873-4da0-9f8b-a7f970c13de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.340153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb382077-7873-4da0-9f8b-a7f970c13de2 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.341374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb382077-7873-4da0-9f8b-a7f970c13de2 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.342849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb382077-7873-4da0-9f8b-a7f970c13de2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.345493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 197/399] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.346731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.347491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.347781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.359679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eae459-de82-4e0d-aee9-a633c1f794ad None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.365721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:41.366988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:41.402267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.403555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.404048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.404527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.406164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.408546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.410551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 203/400] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:41.411601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.445096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 198/401] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:41.445096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.533318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.538111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.552582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7e915-451b-4058-8175-cec20783bcbb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.560836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade67e9-534d-4c6b-be43-ec99281ba295 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.591672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.592621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.592916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.593194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079/groups/3275ae7a3c9b419ca5af63b112f74333/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.596052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8612db90e842afbd07030620e8c079, group_id=3275ae7a3c9b419ca5af63b112f74333, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.597576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.603366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b54a1-e30a-44b0-b62f-dad9318a96de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.604551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b54a1-e30a-44b0-b62f-dad9318a96de None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.604978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b54a1-e30a-44b0-b62f-dad9318a96de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.605767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b54a1-e30a-44b0-b62f-dad9318a96de None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df/groups/3275ae7a3c9b419ca5af63b112f74333/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.608929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b54a1-e30a-44b0-b62f-dad9318a96de None admin] RBAC: Authorizing `identity:create_grant(project_id=3b7e424d403346c5bbf92a367142a6df, group_id=3275ae7a3c9b419ca5af63b112f74333, role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:41.611922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b54a1-e30a-44b0-b62f-dad9318a96de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:41.614750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 199/402] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:41 2022] PUT /identity/v3/projects/4e8612db90e842afbd07030620e8c079/groups/3275ae7a3c9b419ca5af63b112f74333/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:41.615960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.629825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 204/403] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:41 2022] PUT /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df/groups/3275ae7a3c9b419ca5af63b112f74333/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:41.754146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11f48e-5c54-4431-8941-debfeea231fd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:41.758268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4638a-97b3-4bbb-8cd9-419d25691aba None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.758562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4638a-97b3-4bbb-8cd9-419d25691aba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.758853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4638a-97b3-4bbb-8cd9-419d25691aba None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.760588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 205/404] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:29:41.765927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0475e-086c-487d-aa54-de184959155e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.766224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0475e-086c-487d-aa54-de184959155e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.766512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0475e-086c-487d-aa54-de184959155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.800540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:41.801754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:41.802266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:41.802739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:41.810884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-20d0475e-086c-487d-aa54-de184959155e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:41.812164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-20d0475e-086c-487d-aa54-de184959155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:41.817300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:41.826305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:41.827536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:41.828414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:41.831727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 200/405] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:41.844016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.848273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:41.930628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 206/406] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:41.956208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0475e-086c-487d-aa54-de184959155e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:41.985373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5ce57-d368-45d0-a247-151a1d91130c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.034952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.036259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.037497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.038830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.039358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.040407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.040923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.042640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.054319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 201/407] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:42.062647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.080247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0475e-086c-487d-aa54-de184959155e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.117322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.118258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.118506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.118793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.136789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:42.149212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:42.150143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:42.150934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 207/408] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:42.158924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.188337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.301029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868930-7fcf-4c51-b309-fe8579cb9175 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.307635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3aa026-f338-4c7b-ad61-4bc68f0c6b56 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.337853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.338542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.338831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.339447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.340221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:42.340537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.341962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.351215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 208/409] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.352471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.358624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.404931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.405860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.406154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.406437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.408396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.409918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.411643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 202/410] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.421078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.506895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458631b-fe6c-4c31-8317-400b6e1cc69e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.542148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a1496-8575-4848-9e14-c95f8185c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.543018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1496-8575-4848-9e14-c95f8185c53e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.543308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1496-8575-4848-9e14-c95f8185c53e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.543587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1496-8575-4848-9e14-c95f8185c53e None admin] PATH_INFO: `/v3/groups/acfc9696337540dcb078be5560864544` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.544817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a1496-8575-4848-9e14-c95f8185c53e None admin] RBAC: Authorizing `identity:get_group(group_id=acfc9696337540dcb078be5560864544)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.546304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a1496-8575-4848-9e14-c95f8185c53e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.549266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccad19-4fec-4c58-9eca-ded6cb2d1bb5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.553096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 209/411] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/groups/acfc9696337540dcb078be5560864544 => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.554474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa316-a256-447d-a6f5-d19750f86f5f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.554770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa316-a256-447d-a6f5-d19750f86f5f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.555055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa316-a256-447d-a6f5-d19750f86f5f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.556620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 210/412] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:42.557346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa316-a256-447d-a6f5-d19750f86f5f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.584630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963b0fa-0349-4ada-90ec-80960ecdf069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.585634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b0fa-0349-4ada-90ec-80960ecdf069 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.586022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b0fa-0349-4ada-90ec-80960ecdf069 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.586455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b0fa-0349-4ada-90ec-80960ecdf069 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.587803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963b0fa-0349-4ada-90ec-80960ecdf069 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.589432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963b0fa-0349-4ada-90ec-80960ecdf069 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.591712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 203/413] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.592078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.597276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.597587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.597930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.616490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:42.617737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:42.633113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.678365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa316-a256-447d-a6f5-d19750f86f5f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.692525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 204/414] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:42.693812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.716317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.717242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.717589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.717892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] PATH_INFO: `/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.719249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] RBAC: Authorizing `identity:get_project(project_id=10e451bce05f4f3c975f7e8a5cfb1440)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.721697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.724019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 211/415] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:42.729510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.802706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb631c1-e9b4-4127-b22f-aaef8a21a3fd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.833592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.840175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.841256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.841719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.842141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079/groups/3275ae7a3c9b419ca5af63b112f74333/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.842750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6ce70-d585-4a89-bf32-726c8492d0cd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:42.844742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8612db90e842afbd07030620e8c079, group_id=3275ae7a3c9b419ca5af63b112f74333, role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:42.847125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:42.862525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 205/416] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:42 2022] PUT /identity/v3/projects/4e8612db90e842afbd07030620e8c079/groups/3275ae7a3c9b419ca5af63b112f74333/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:42.863618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:42.888280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:42.902648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:42.905122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:42.905598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:42.908388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:42.936563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:42.952331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:42.953521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:42.955022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:42.961150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 212/417] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:42.965542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.088818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571078bc-7186-4bdd-9156-3fae753beb55 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.131038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e579a-4a9f-4e27-bdba-81e14b471d1e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.139661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.140571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.140859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.141140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] PATH_INFO: `/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440/groups/acfc9696337540dcb078be5560864544/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.143824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=10e451bce05f4f3c975f7e8a5cfb1440, group_id=acfc9696337540dcb078be5560864544, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.145266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.163561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 206/418] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Jun 15 14:29:42 2022] PUT /identity/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440/groups/acfc9696337540dcb078be5560864544/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:29:43.203762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.204995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.205288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.205569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.206496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:43.206814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.208255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.224430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 213/419] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:43.229984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.289153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.366821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e306-9441-4dd0-a02d-3c15fc2446d2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.403937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.404603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.404892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.405175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.475769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.477286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.529606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.584430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 207/420] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:43.590056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.657478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:43.707046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e18be-ebd3-4170-9d4d-e31f2ccff248 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.742319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.743199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.743490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.743771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.744985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.746451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.748457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 214/421] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:43.754652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:43.891372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0a95-b4c9-407a-9fa7-e0a1eae19da5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:43.933398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a355749-576a-4e62-bb8b-f607673749c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:43.934218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a355749-576a-4e62-bb8b-f607673749c5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:43.934510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a355749-576a-4e62-bb8b-f607673749c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:43.934793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a355749-576a-4e62-bb8b-f607673749c5 None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:43.938362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a355749-576a-4e62-bb8b-f607673749c5 None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:43.939982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a355749-576a-4e62-bb8b-f607673749c5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:43.941978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 208/422] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:43.947675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.063347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ed52-2cfb-4c3b-9fc9-823599135134 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.085352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.107336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.108017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.108310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.108594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.109449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=3b7e424d403346c5bbf92a367142a6df) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.109796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.111386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675dce7-166e-4141-8bba-1a2556e7fe67 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.122357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 215/423] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:43 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.246073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8b16-c31d-4dc2-b2fd-ee8f6c665638 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.246689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8b16-c31d-4dc2-b2fd-ee8f6c665638 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.247191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8b16-c31d-4dc2-b2fd-ee8f6c665638 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.249098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 209/424] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:44.255528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.256141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.256651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.275966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:44.277600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:44.347455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356016a-f7fa-489b-a348-276b0b047b2e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.347794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356016a-f7fa-489b-a348-276b0b047b2e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.348033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356016a-f7fa-489b-a348-276b0b047b2e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.349631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 210/425] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:44.355831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.355831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.355831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.376650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:44.378360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:44.391041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 216/426] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:44.396093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.419060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.483741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 211/427] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:44.518448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.549981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e3f21-a2ed-4ef8-861b-e28c431a70a4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.566092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.598396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.599237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.599532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.599817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.615754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:44.625210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:44.626750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:44.628666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 217/428] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:44.634352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.639380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e8a94-be5a-44e9-abd9-5aafc3c69012 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.698061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.699292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.699758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.700392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.713645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:44.722840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:44.723769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:29:44.724590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:44.727301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 212/429] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:44.732910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.747890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.788431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada55e17-6495-4089-9233-f849b5af5edc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.827252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.828635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.828635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.828635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.829485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.830044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.831288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.840255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 218/430] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.846808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:44.887905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:44.902191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f412-62e6-4650-8084-cec899b01b4b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.944231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:44.945285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:44.945905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:44.946489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:44.947500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:44.948014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:44.949780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:44.962390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 213/431] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:44.967018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2084ea4-8b50-419f-a977-b29b95034812 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:44.972154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.004430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.006207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.007231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.007725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.060667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.100721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf17bf-8d03-4c06-a8c8-8c37002383ee None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.105054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.106803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.141314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.142647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.142907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.143189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.152860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.228655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 219/432] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:45.228838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.228838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.230175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.308251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.332618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000a6f-28b5-4ee5-8654-ea0470f88e62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.347544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 214/433] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:45.353450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.368435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.369331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.369617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.369929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] PATH_INFO: `/v3/projects/3b7e424d403346c5bbf92a367142a6df` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.371250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] RBAC: Authorizing `identity:get_project(project_id=3b7e424d403346c5bbf92a367142a6df)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.372747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.379844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 220/434] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/projects/3b7e424d403346c5bbf92a367142a6df => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:45.381452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.385401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.479603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b156e29-4d60-4048-a885-23135d9493c9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.487237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20831ff5-c7ff-40f2-a25f-95dce5f50e85 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.516044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35c645-e3f1-47ac-88b4-05d238e58d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.516978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35c645-e3f1-47ac-88b4-05d238e58d0a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.517242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35c645-e3f1-47ac-88b4-05d238e58d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.517524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35c645-e3f1-47ac-88b4-05d238e58d0a None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.519128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35c645-e3f1-47ac-88b4-05d238e58d0a None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.520554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35c645-e3f1-47ac-88b4-05d238e58d0a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.522312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 215/435] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:45.523494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7554cc3-b5c2-4eb5-af3f-a060ca2d51c0 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.523780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7554cc3-b5c2-4eb5-af3f-a060ca2d51c0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.524057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7554cc3-b5c2-4eb5-af3f-a060ca2d51c0 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.525437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 216/436] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:45.528310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200620a7-8a50-4b1e-bc69-8f028a0b6513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.529513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200620a7-8a50-4b1e-bc69-8f028a0b6513 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.529986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7554cc3-b5c2-4eb5-af3f-a060ca2d51c0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.530158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200620a7-8a50-4b1e-bc69-8f028a0b6513 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.530694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200620a7-8a50-4b1e-bc69-8f028a0b6513 None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.532042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200620a7-8a50-4b1e-bc69-8f028a0b6513 None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.534316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200620a7-8a50-4b1e-bc69-8f028a0b6513 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.539999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 221/437] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:45.541483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.542285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.542573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.560303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:45.561496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:45.573629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.638003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7554cc3-b5c2-4eb5-af3f-a060ca2d51c0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.653681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 222/438] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:45.653681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.654918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.676762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.677629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.677960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.678241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.679419 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.680848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.682635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 217/439] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:45.686916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.765871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad12dcd-306e-40e7-9fbf-e2ed6626d253 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.796155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a38cc-0b40-4dea-8e8c-87958b95ecef None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.804706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.805143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.805843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.806135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.806424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.807283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=3b7e424d403346c5bbf92a367142a6df) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:45.807786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:45.809476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:45.816672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 223/440] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=3b7e424d403346c5bbf92a367142a6df&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:45.817569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.836752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.838048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.838550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.852730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:45.862340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:45.863371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:45.864192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 218/441] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:45.869545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:45.919082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:45.960556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b99fb4-b24e-4bf7-b8d3-3e40b08a04f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.992495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c165670-3f6a-486e-8775-461b064717eb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:45.997549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:45.997892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:45.997892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:45.998175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:45.999114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=bdd97d4caedc488bac31aec6d879f431, scope.project.id=4e8612db90e842afbd07030620e8c079) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:45.999371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.000774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5fc08-4ad0-4a7f-a31c-931987bc3538 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.008602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 224/442] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=bdd97d4caedc488bac31aec6d879f431 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:46.010003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f55a3d-f669-49aa-bc75-f30c1aa68ead None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.010280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f55a3d-f669-49aa-bc75-f30c1aa68ead None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.010557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f55a3d-f669-49aa-bc75-f30c1aa68ead None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.011970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 225/443] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:46.024225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.024225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.024225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.037547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:46.038801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:46.042516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.043269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.043570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.043859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.044651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:46.044971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.046435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.057900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 219/444] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:46.063196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.111565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.135008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 226/445] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:46.159982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.208569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbb6e9-3d40-4928-a436-9e133ded5473 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.228809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.262282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.263105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.263398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.263683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.281816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594314a7-1bd2-4dc0-827b-d00c4e8e28b4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.321939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.323133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.323424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.323706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.336304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:46.345706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:46.346613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:46.347430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 227/446] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:46.351333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.352316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.353849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.384860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.470575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b9c2-d1ab-492f-9a1f-f549c8bbe607 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.471453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 220/447] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:29:46.477019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.491527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.505927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.506604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.506891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.507177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.507925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:46.508241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.509616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.519057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 228/448] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:46.524435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.575040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcbc82-1516-4d0d-bebe-2f42ed61c252 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.601036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.609567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.610505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.610788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.611072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] PATH_INFO: `/v3/projects/4e8612db90e842afbd07030620e8c079` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.612259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8612db90e842afbd07030620e8c079)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.613704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.615403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 221/449] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/projects/4e8612db90e842afbd07030620e8c079 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:46.620716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.623827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d914c6-7f40-4f0d-b244-1054062dc705 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.657935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.658589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.658884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.659167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.708623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.721350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86882418-5f65-422a-92ae-ce264d550f04 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.727420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.728890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.755464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.756320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.756608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.756888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] PATH_INFO: `/v3/groups/3275ae7a3c9b419ca5af63b112f74333` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.758087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] RBAC: Authorizing `identity:get_group(group_id=3275ae7a3c9b419ca5af63b112f74333)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.759512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.761132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 222/450] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/groups/3275ae7a3c9b419ca5af63b112f74333 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:46.767216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.829940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.834411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 229/451] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:46.839907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:46.868753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e1b7e-bc60-4f21-9019-249c08179dcb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.902196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.902854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.903140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.903430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.904262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] RBAC: Adding query filter params (group.id=3275ae7a3c9b419ca5af63b112f74333, role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=4e8612db90e842afbd07030620e8c079) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:46.904580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.906031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.911892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 223/452] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/role_assignments?group.id=3275ae7a3c9b419ca5af63b112f74333&scope.project.id=4e8612db90e842afbd07030620e8c079&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:46.925978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:46.939453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f44a2-2890-46ae-aef5-5c7547c9054d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:46.975870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:46.976668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:46.976953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:46.977233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] PATH_INFO: `/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:46.978481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] RBAC: Authorizing `identity:get_project(project_id=10e451bce05f4f3c975f7e8a5cfb1440)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:46.979976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:46.981762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 230/453] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/projects/10e451bce05f4f3c975f7e8a5cfb1440 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:46.988124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.093502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927bd1d-9b5e-4337-9918-77ca10179394 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.121153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:47.129683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d219d7-fdeb-4e52-8c77-4be023c48613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.130664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d219d7-fdeb-4e52-8c77-4be023c48613 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.130953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d219d7-fdeb-4e52-8c77-4be023c48613 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.131237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d219d7-fdeb-4e52-8c77-4be023c48613 None admin] PATH_INFO: `/v3/groups/acfc9696337540dcb078be5560864544` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.132426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d219d7-fdeb-4e52-8c77-4be023c48613 None admin] RBAC: Authorizing `identity:get_group(group_id=acfc9696337540dcb078be5560864544)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.133964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d219d7-fdeb-4e52-8c77-4be023c48613 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.135598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 224/454] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Jun 15 14:29:46 2022] GET /identity/v3/groups/acfc9696337540dcb078be5560864544 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:47.141330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:47.259457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9496c3-f01e-4f79-973b-e747c052eb96 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:47.300726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:47.301399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:47.301743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:47.302031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:47.302882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] RBAC: Adding query filter params (group.id=acfc9696337540dcb078be5560864544, role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=10e451bce05f4f3c975f7e8a5cfb1440) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:47.303204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:47.304645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982959bf-e476-4efc-80c8-ac03a61d6b76 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:47.310829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 231/455] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Jun 15 14:29:47 2022] GET /identity/v3/role_assignments?group.id=acfc9696337540dcb078be5560864544&scope.project.id=10e451bce05f4f3c975f7e8a5cfb1440&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:49.842027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564d36f-1ec2-445c-854e-3e842b9ea081 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:49.842576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564d36f-1ec2-445c-854e-3e842b9ea081 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:49.842576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564d36f-1ec2-445c-854e-3e842b9ea081 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:49.844256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 225/456] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:49.850007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a91921-9292-422b-a947-51090a32ceda None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:49.850459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a91921-9292-422b-a947-51090a32ceda None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:49.851948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a91921-9292-422b-a947-51090a32ceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:49.874678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f9a91921-9292-422b-a947-51090a32ceda None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:49.876702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9a91921-9292-422b-a947-51090a32ceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:49.887221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:49.964943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 232/457] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:49.998227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564d36f-1ec2-445c-854e-3e842b9ea081 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.058605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38f3c1-4448-4bed-847a-baa330b5a0ca None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.059130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38f3c1-4448-4bed-847a-baa330b5a0ca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.059557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38f3c1-4448-4bed-847a-baa330b5a0ca None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.061341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 233/458] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:50.066693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.067199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.067624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.111575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:50.112704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:50.155637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564d36f-1ec2-445c-854e-3e842b9ea081 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.223860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.224786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.225062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.225342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.229464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 234/459] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 15 14:29:50.241115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:50.242878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:50.262100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:50.263281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:50.264304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:50.265346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:50.265346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:50.265346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:50.265346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:50.267674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 226/460] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:50.272905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.350037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.406862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10babfb7-7948-4eb2-baa7-ed3a66c83e1e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.472530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.473474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.473967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.474387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.475271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:50.475727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:50.477302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381e05-2d41-4c91-ac65-1cd0f55a0750 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:50.496288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.501220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 227/461] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:50.501220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e99a7-097a-4a06-8576-879d9e837f5b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.501220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e99a7-097a-4a06-8576-879d9e837f5b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.501220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e99a7-097a-4a06-8576-879d9e837f5b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.502326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 228/462] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:50.503182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e99a7-097a-4a06-8576-879d9e837f5b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.530161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46b316-6966-4a21-b2bc-c2371c6a7de6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.564193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.564995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.565283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.565563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.577134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:50.578624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-490d7975-897e-4fb6-9cff-8b4e79b9ec77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:50.599135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:50.600020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 235/463] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370ec7-442e-4647-83d7-74dfaf54de4a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370ec7-442e-4647-83d7-74dfaf54de4a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.606461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370ec7-442e-4647-83d7-74dfaf54de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.618777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-84370ec7-442e-4647-83d7-74dfaf54de4a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:50.619967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-84370ec7-442e-4647-83d7-74dfaf54de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:50.620618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e99a7-097a-4a06-8576-879d9e837f5b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.632625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.661164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b62416-1e25-49cb-b990-2c37aeb5ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.661799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b62416-1e25-49cb-b990-2c37aeb5ec5c None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.662094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b62416-1e25-49cb-b990-2c37aeb5ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.662403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b62416-1e25-49cb-b990-2c37aeb5ec5c None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.663119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b62416-1e25-49cb-b990-2c37aeb5ec5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:50.664549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b62416-1e25-49cb-b990-2c37aeb5ec5c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:50.685747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 229/464] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:50 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:50.691491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33a11-582d-44dd-a5bf-502ca9e282e9 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.691850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33a11-582d-44dd-a5bf-502ca9e282e9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.692147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33a11-582d-44dd-a5bf-502ca9e282e9 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.693431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 236/465] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:50.702447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 230/466] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:50.702447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33a11-582d-44dd-a5bf-502ca9e282e9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.709176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.709493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.709797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.738408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:50.739644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:50.799712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.825867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 237/467] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:50.827057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.858005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33a11-582d-44dd-a5bf-502ca9e282e9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:50.897351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:50.898072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:50.898361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:50.898641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:50.899368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:50.899681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:50.901037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:50.916571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 231/468] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:50.918594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:50.949755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:50.958075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c2021-9c7a-4637-9526-7e6b389a5864 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.004385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.005206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.005501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.006047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.017208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.018258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:51.039958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:51.040870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:51.041779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:51.041779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:51.041779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:51.041779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:51.041779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 238/469] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:51.042556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.061069 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a9330-232f-4e6d-81ad-e258d870e91b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.085987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.112308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.113121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.113415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.113767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.124562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.126124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:51.145124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:51.146153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:51.146991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:51.146991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:51.146991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:51.146991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:51.146991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 232/470] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:51.152801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.167459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32296b00-d07d-40bb-a7ce-a18483e3e7de None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.206459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.207192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.207487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.207771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.208492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.210018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.233134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 239/471] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:51.234209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.256029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.283420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231fa03-7d02-403a-9161-1c83a80a5b8d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.320471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.321208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.321439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.321752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.322497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:51.322820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.324312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.338616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 233/472] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:51.343724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.372794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.393117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb4eea-5d5f-496d-8da0-525c49ec51b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.477439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.480302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.480662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.480965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.481750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:51.482077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.483500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd6791-b17a-4bce-89fa-6d3862ad246d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.505385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 240/473] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:51.506829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550e71-0bfd-4e25-8cd4-4d643d356a61 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.507124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550e71-0bfd-4e25-8cd4-4d643d356a61 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.507408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550e71-0bfd-4e25-8cd4-4d643d356a61 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.511698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 241/474] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:51.513755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550e71-0bfd-4e25-8cd4-4d643d356a61 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.531982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68048bd7-9dab-46f5-a751-95a4c9bfc67e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.578197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c56567-bd99-44c2-acdf-cc93001d8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.578875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c56567-bd99-44c2-acdf-cc93001d8a0f None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.579290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c56567-bd99-44c2-acdf-cc93001d8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.579574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c56567-bd99-44c2-acdf-cc93001d8a0f None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.580300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c56567-bd99-44c2-acdf-cc93001d8a0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.581767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c56567-bd99-44c2-acdf-cc93001d8a0f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.589576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.621778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 234/475] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:29:51.623363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.623661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.623944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.668183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:51.671162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:51.684008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550e71-0bfd-4e25-8cd4-4d643d356a61 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:51.769988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:51.771041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:51.771795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:51.773078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:51.774293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:51.777099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:51.810042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 242/476] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/users => generated 249 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:51.820253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 235/477] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:51.844342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:51.859896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:51.965936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce7ef6-8830-4110-ac19-a96eb0e01348 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.003880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.004573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.004865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.005155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.005953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.007382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.014190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.sql.core [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:29:52.014190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:29:52.014190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [parameters: {'id': 'f17d2aee355945ec8235cc96cd5496a1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:29:52.014190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80378) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Jun 15 14:29:52.032683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:29:52.033637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application yield resource Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise self.value Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application session.commit() Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:29:52.035123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:29:52.036245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:29:52.037786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:29:52.039980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [parameters: {'id': 'f17d2aee355945ec8235cc96cd5496a1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:52.042073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:29:52.045387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:52.047472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 243/478] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Jun 15 14:29:51 2022] POST /identity/v3/roles => generated 144 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:52.047607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.163057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9bccc-2386-4e50-b7a7-bcb4175f1c3b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.189530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.203332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee0f4d-ebc5-47fb-a306-6ac0e85070a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.204537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0f4d-ebc5-47fb-a306-6ac0e85070a0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.205064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0f4d-ebc5-47fb-a306-6ac0e85070a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.205536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0f4d-ebc5-47fb-a306-6ac0e85070a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.216441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee0f4d-ebc5-47fb-a306-6ac0e85070a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-20ee0f4d-ebc5-47fb-a306-6ac0e85070a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:52.225101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:52.226036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:29:52.226833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:52.229648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 236/479] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:52.237687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:52.328578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:52.359327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447d2e-85b9-45bd-82eb-6d1bcaf5a3f0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:52.394470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:52.395157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:52.395450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:52.395733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:52.396495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:52.396811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:52.398301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decde69a-43ce-432b-b009-4f11736c7ad3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:52.407761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 244/480] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.087987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be2bd-3265-458c-84fd-48a0a5b5859a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.088525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be2bd-3265-458c-84fd-48a0a5b5859a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.088604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be2bd-3265-458c-84fd-48a0a5b5859a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.090239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 237/481] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:53.096109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.096502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.096684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.114614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:53.115808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:53.187294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 245/482] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:53.210758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be2bd-3265-458c-84fd-48a0a5b5859a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.330365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be2bd-3265-458c-84fd-48a0a5b5859a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.373630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.374790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.375277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.375751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.395313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:53.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:53.417342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:53.418172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:53.425146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 238/483] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:53.427280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.547087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.593510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f53ae3-62d2-4c6b-ab6d-fd4ec9cc6329 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.644034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.644638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.644930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.645220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.646009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:53.646331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.647709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb4fa9-1e3b-4120-8c69-deaafe39aad7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.661632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 246/484] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:53.666850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.762194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:53.833299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38585393-26b7-497f-86e0-8546be943f04 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:53.857992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac8226-1641-44c3-92f2-c8d94cdff661 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.858291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac8226-1641-44c3-92f2-c8d94cdff661 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.858580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac8226-1641-44c3-92f2-c8d94cdff661 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.860210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 247/485] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:53.865304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.865703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.865916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.868987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:53.870031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:53.870421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:53.870806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:53.883280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:53.883447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:53.884610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:53.885046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:53.906651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:53.907999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:53.909138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:53.909138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:53.909138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:53.909138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:53.911185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 239/486] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:53.918608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:53.998103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 248/487] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:54.048142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.098078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.115673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162002a6-f7fb-444a-9b62-9bf4dfd75daf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.151377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.179582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.180182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.180474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.180760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.181509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.181853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.183239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.209615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 240/488] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.210552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.234927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ee3ad-2b29-4740-b1fb-e166c8e1b168 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.300789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.301674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.301973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.302257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.320031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:54.336762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.337912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:54.338976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:54.339574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 249/489] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.347459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.405107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3e45d-70a3-4aad-ae92-4f99166b204e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.471967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.472800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.473090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.473372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.486973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.488465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:54.524828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.525818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:54.526700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 241/490] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.531878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.548501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.559585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669814-46e2-4dd9-b5a6-216a68efeb8b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.580582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.599616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.600402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.600694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.600977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.601760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.602068 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.603578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3de964-0870-403b-8979-4b9f7ebef770 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.613001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 250/491] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.617381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae70b4-85e2-4942-90b4-8f215fef7843 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.617381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae70b4-85e2-4942-90b4-8f215fef7843 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.617381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae70b4-85e2-4942-90b4-8f215fef7843 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.620114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 251/492] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:54.620896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae70b4-85e2-4942-90b4-8f215fef7843 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.650840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896491f-98f2-4526-be08-ed8cec4e6395 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.687480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.688237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.689192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.689358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.690238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.690550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.692053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b9759-749f-4b2d-97a0-4d7c25097acd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.735981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 242/493] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:54.738828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ed55e-bac9-4ca5-aade-2272a976e37f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.738828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ed55e-bac9-4ca5-aade-2272a976e37f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.738828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ed55e-bac9-4ca5-aade-2272a976e37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.750500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae70b4-85e2-4942-90b4-8f215fef7843 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.757695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eb9ed55e-bac9-4ca5-aade-2272a976e37f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:54.757695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eb9ed55e-bac9-4ca5-aade-2272a976e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:54.790923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.791832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.792122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.792403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.805496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.807015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:54.825585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:54.826950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:54.827817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:54.827817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:54.827817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:54.827817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:54.827817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 252/494] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:54.828082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.832681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 243/495] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:54.834327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b554f-7e90-40c6-923e-b764cd87eaf6 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.834843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b554f-7e90-40c6-923e-b764cd87eaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.835320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b554f-7e90-40c6-923e-b764cd87eaf6 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.841085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 244/496] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:54.841085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b554f-7e90-40c6-923e-b764cd87eaf6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:54.845524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.867261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:54.949078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc18090-1dc0-4c16-a84f-1726fa5062cd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.953679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b554f-7e90-40c6-923e-b764cd87eaf6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:54.984841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.986638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.986918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.987221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.990949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:54.991621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:54.991912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:54.992873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:54.992987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:54.993226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:54.994625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2272bf-efff-4a19-881c-6047cb5b97e1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.007092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 245/497] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.008143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.008440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.008726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.024784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:55.026014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:55.066410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.067876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.086616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.097955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 246/498] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:55.098699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.098810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.197530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 253/499] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:55.197530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.212963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3756e-66c3-4a79-9e98-53b64638f858 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.247869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.248720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.249063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.249412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.260758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:55.269133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:55.270224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:55.271017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 247/500] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:55.271675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.298658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da07868-cfd4-4953-ae69-347a2e9f6b6a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.326325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.328430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.334414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.335222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.335516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.335796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.342745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.344208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:55.361172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:55.362308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 254/501] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:55.363203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.379304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da8c5-a844-4dbe-8285-1d7c0087e865 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.412656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb9206-48b5-4f3d-bb2f-4d1b761d1a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.413451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9206-48b5-4f3d-bb2f-4d1b761d1a4e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.413776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9206-48b5-4f3d-bb2f-4d1b761d1a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.414067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9206-48b5-4f3d-bb2f-4d1b761d1a4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.424891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb9206-48b5-4f3d-bb2f-4d1b761d1a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-83bb9206-48b5-4f3d-bb2f-4d1b761d1a4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:55.432754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:55.433683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:55.434498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 248/502] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:55.435500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac7f2d-77ae-423d-aa76-74b641aae354 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.435500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac7f2d-77ae-423d-aa76-74b641aae354 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.435778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac7f2d-77ae-423d-aa76-74b641aae354 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.437183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 249/503] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:55.437855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac7f2d-77ae-423d-aa76-74b641aae354 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.465347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adee217-6d2a-41ec-a7b8-7e4ab33507c8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.499520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.500241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.500525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.500894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.511040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.512623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:55.530157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:55.531170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:55.532558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:55.532558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:55.532558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:55.532558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:55.532558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 255/504] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:55.533199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.543733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac7f2d-77ae-423d-aa76-74b641aae354 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.556762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.578311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.579099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.579441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.579605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.579766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.580523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.580833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.582339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.590259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 250/505] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.591060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.637500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510b3f-dee5-4a9f-b076-46ee7a47d2ac None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.670799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.671474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.671756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.672034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.673448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.673799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.675225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd3b74-0c3b-49f6-a34d-b1dd44615a13 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.692589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29536a1e-0de4-43d3-b550-04818a0786c0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.711196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 256/506] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.712450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.712747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.713028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.727924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.728597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.728887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.729142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:55.729240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.729951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.730271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.730358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:55.731632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.739699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 251/507] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.740482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.774735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.801084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 257/508] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:29:55.802157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.832806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.846629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52203a-b97c-42df-a430-f20efcfc2dab None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.879621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.880282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.880598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.880879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.881613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:55.881961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.883344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.895965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 252/509] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:55.896598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.902469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d821612-a596-4a23-8a4e-12afe2f28164 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:55.938656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:55.939625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:55.939911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:55.940192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:55.950398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:55.952113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:55.971227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:55.972274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:55.973334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:55.973334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:55.973334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:55.973334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:55.973947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 258/510] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:55.974755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:55.989779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:55.996165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf6140-1431-44c1-8a64-2c83ccbeb26e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.029542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.030226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.030516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.030800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.057833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.074878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924c512-e159-4c0f-9573-5b05e2406cb5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.098531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.100012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.108271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.109065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.109357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.109635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.120080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.121517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:56.138888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.139895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.146559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:56.146559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:56.146559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:56.146559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:56.146559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 259/511] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:56.146559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.192421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.200232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 253/512] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:56.201200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.242898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b51de-74b0-48f8-9451-e9485bf125be None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.275267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.276068 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.276358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.276740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.279419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.288706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.290166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.299161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda6368-4234-4d97-ab78-1ca64e2fd5f2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:56.307644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.308520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.309354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:56.309354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:56.309354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:56.309354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:56.309354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 260/513] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:56.309613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.337297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.338127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.338418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.338700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.346036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.347498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:56.362318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.363162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:56.364020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 254/514] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:56.364478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.412938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02c995-788f-4c3e-94ca-fc7994695334 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.419106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.449161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.449161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.449161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.449161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.449574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.449574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.450676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.464309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 261/515] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.465131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.467917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8f1a7-4533-451d-a24b-4932ea36e70e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.504158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.504939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.505226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.505511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.506593 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.507974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.513813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.521111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 255/516] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.521960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.569357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e90a9c-88c8-441a-8f7e-1224755dece4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.603254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.604049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.604338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.604616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.614345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.615794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.618148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d2d26-b8cd-4796-99b9-7ba5926b0bfd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:56.633184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.634186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.635297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:56.635297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:56.635297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:56.635297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:56.635579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.635650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 262/517] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:56.635650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.657708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.658756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.659044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.659333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.660065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.660386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.661780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.672748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 256/518] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.673625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.732971 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.736010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a70028-601b-47c1-bd96-a07d2b889691 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.769804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.770480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.770774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.771057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.771927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.772244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.773719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.774257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40726-5190-401b-bca4-25dcb6e9c9b0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.792565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 263/519] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.793416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.816864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.817689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.818007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.818293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.825906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.827543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:56.842693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.843601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:56.844539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 257/520] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:56.846004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.879487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.901374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044076-cbdf-4732-8081-cfd34e14a3ae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.934915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.935719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.936006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.936285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.943568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7a6c5-5207-4147-8002-2a1d0a365037 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:56.944487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.945970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:29:56.961914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:56.962828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:56.963740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 264/521] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:56.964421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:56.978834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:56.979503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:56.979792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:56.980073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:56.980804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:56.981118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:56.982503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:56.988896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:56.998753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 258/522] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:56.998753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.068928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f544a9-af86-444b-aa0d-0392dde5bd0f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.092364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.098948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44991b-f021-4015-83e0-4308c3116d23 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.107222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.107901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.108200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.108482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.109196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.110921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.131755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 265/523] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:29:56 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:29:57.132586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.136224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.137084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.137432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.137806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.138738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=f5953aa40eb54d399b593000f3c74596) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:57.139132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.140682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.158098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 259/524] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:56 2022] GET /identity/v3/role_assignments?user.id=f5953aa40eb54d399b593000f3c74596&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:57.158098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.250741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.281723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d315c-e129-4ee3-bc71-f74c912b6fa0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.315547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae299d63-004a-41a6-89c1-218e3ba251d5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.323266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.323931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.324226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.324510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.325386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:57.325746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.327639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.355775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 266/525] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:57.357249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.370221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.370921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.371215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.372153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.372485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:57.372749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.374287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.395133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 260/526] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:57.396201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.418763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.485977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25dd36-635a-4819-ba31-7abcaa1b697e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.523892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.524794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.524977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.525261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.527589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.534590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.536656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.541030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e876a-07ed-4cbf-9f1b-b3bd3bbdbcc9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:57.552558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:57.553468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 267/527] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:57.555106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.591163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.591834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.592122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.592406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.669315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def63518-2dbf-4398-b2e7-7a35596eaf81 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.685938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.688738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.708867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.710264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.711211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.711818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.712293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.795602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.807178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.808925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.810863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 261/528] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:57.812082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.916979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc42a8-c2cd-4bf7-88e3-e426b6e92a88 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:57.943662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 268/529] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:29:57.943662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:57.954141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:57.963320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:57.964473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:57.965240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:57.965751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:57.966773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:57.967286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:57.970138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:57.999997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 262/530] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:57.999997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.063860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.073019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95298354-55df-4bad-8e66-aa23818502d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.115715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e50a6e-f2ad-400f-a06e-819d4e5654e8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.121802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.122843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.123405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.123883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.139419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.142054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.153507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.154384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.154677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.154957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.165931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.167370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:58.167835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.168676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.169518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:58.169518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:58.169518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.169518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:58.176584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 269/531] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.176584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:29:58.197074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.197983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.198835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:29:58.198835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:29:58.198835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:29:58.198835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:58.203114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 263/532] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.203879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.312595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b322d-5484-401c-a020-b5224e2fe55b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.319568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1973f8-cd46-46fb-961b-5d8836bbfbe1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.351923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.352599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.352889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.353172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.354016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.354269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.355710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.373962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 264/533] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:58.373962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.377391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.378092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.378385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.378669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.380245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=3ca2cd515aa84a3ca01656977036ac07) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.380245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.381268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.393818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 270/534] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/role_assignments?user.id=3ca2cd515aa84a3ca01656977036ac07&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:58.394839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.424398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.501786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.516864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9241e6-43b8-4866-aacf-8d9ebd0074d0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.574493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.575197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.575495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.575783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.576538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.577116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.579115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.601576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 265/535] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:58.606919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.607386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563602d-d9bb-400d-aaa4-4669666041e6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.645079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.646276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.646764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.647236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.654865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.656649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:58.672145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.673640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:58.674533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:29:58.677069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 271/536] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.682801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.713038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f4f2-7338-47e3-b1a8-4e469860dc34 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.748251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.749055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.749350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.749628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.756833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.758333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.762098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:29:58.773885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:58.774794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:58.775674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 266/537] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:58.780892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.794895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:58.805437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd6a8-4511-42f9-9704-36b167e90482 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.842991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.844085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.845248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.845758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.846767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.847402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.849156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.863245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 272/538] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:58.869346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:58.888203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cf4b5-eeca-409d-8cf9-9cf770b50bfb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:58.952670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:58.955542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:58.956314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:58.958918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:58.959952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:58.961921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:58.963642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:58.990158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 267/539] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:58.997424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.028877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a749-9e88-4a25-b343-16b5cf0d18e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.091805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.101636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.102776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.104322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.104806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.106251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=d292189b741b40b8859bfe9883b4c5ec) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.107237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.109510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b3673-5996-4811-9349-ca575a0cbb15 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.122162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 273/540] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/role_assignments?user.id=d292189b741b40b8859bfe9883b4c5ec&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:29:59.125058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.159574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1517be2-5e24-427c-af06-aef97d2d9987 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.210610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.211681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.212179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.212684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.213896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=a26d10743fe844a98ecfcae3974dbce4) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:29:59.214434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:29:59.216148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785d418-d483-43a8-819f-93adcd2591da None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:29:59.226516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 268/541] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:29:58 2022] GET /identity/v3/role_assignments?user.id=a26d10743fe844a98ecfcae3974dbce4&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:29:59.568238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206ac2c-e7c6-4686-a240-b7ca6cd5fae0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.569258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206ac2c-e7c6-4686-a240-b7ca6cd5fae0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.569786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206ac2c-e7c6-4686-a240-b7ca6cd5fae0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.571577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 274/542] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:59.577596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.578158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.578658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.593353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcdd7e-7342-45bf-b745-f23193afff37 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.593636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcdd7e-7342-45bf-b745-f23193afff37 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.593949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcdd7e-7342-45bf-b745-f23193afff37 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.596497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 275/543] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:29:59.599161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:59.600641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:59.601865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.602162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.602446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.620104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:29:59.621309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:29:59.684035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 269/544] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:59.708763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.748639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 276/545] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:29:59.775834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:29:59.790522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.819643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b565be-473b-4dcf-a9a1-398fe1acd38c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:29:59.886459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:29:59.887660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:29:59.888171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:29:59.888649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:29:59.910756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:29:59.919052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:29:59.919953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:29:59.920749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:29:59.924026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 270/546] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:29:59.931521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:29:59.998238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207171d3-f8ea-4cc6-94df-8b7991292f1b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.054009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.075034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.075784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.076075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.076353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.104953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71a78-39ee-4b22-8020-8e531f6cd414 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.106269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:00.118342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.119287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:00.120088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:00.124216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 277/547] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:00.125314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.134031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.146480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.147165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.147454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.147739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.148507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.148822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.150238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.174847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 271/548] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:29:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:00.191667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.300824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b4ae-eae9-4c4c-9d9a-dafd62f350fe None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.338799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.339528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.339822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.340103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.351829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.353295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:00.372461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.373852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.374883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:00.374883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:00.374883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:00.374883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:00.378101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 272/549] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:00.380418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.404356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418637a5-23bb-4239-acb0-8220b6493155 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.434079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.497337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd30d18-ff22-4bab-808b-472cc7236dbf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.524288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.525035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.525328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.525612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.526413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.526730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.528192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.537146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.537853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.538147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.538449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.539179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:00.539500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.540880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.549984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 278/550] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:00.555159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 273/551] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:00.555387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.560397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.633011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.663167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:00.670933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b95b18-fc91-415f-91c1-5edbe1198d21 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.732505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b1cd9-56a1-49fb-a8a0-29c535da394f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:00.752892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.754809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.755480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.757090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.773043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.774823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.799486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:00.800636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:00.801151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:00.801748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:00.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.812440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:00.813398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:00.816715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 274/552] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:00.820971 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:00.820971 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:00.827252 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:00.839419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:00.840334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:00.841188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:00.841188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:00.841188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:00.841188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:00.843247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 279/553] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:00.849758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.023087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.027205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6d77-791c-422f-87a2-2c87c8ee49c3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.041418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce95d3-7084-4717-86a4-4e314de6cfcd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.065279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.065980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.066274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.066553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.067282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.067594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.069168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d1e7b-23e9-4adc-9cd0-a3225b4a7b42 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.081505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.082467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.082934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.083410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.083777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 280/554] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:01.084637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.085059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52f9af-e215-41e8-8613-b1edcca6c674 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.085317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.085430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52f9af-e215-41e8-8613-b1edcca6c674 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.085610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52f9af-e215-41e8-8613-b1edcca6c674 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.087211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b7f3c-50f3-49cd-9d89-44048336aedf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.096141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 281/555] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:01.096141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52f9af-e215-41e8-8613-b1edcca6c674 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.114187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 275/556] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:01.115921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.116423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.116991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.134785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:01.136356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:01.149141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.208291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52f9af-e215-41e8-8613-b1edcca6c674 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.218041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 276/557] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:01.219247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.247297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.248416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.248829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.249232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.258201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.259975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:01.278480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.279907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:01.280859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:01.284966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 282/558] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:01.290438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.311558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.348587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b621566-98de-4e95-8c7f-6f0e1ff5b56f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.387765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.388438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.388727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.389012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.408065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.408711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2f6ca-ee03-46be-be76-d7e26080889e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.454698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.455975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.456490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.456960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.465324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.466932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.469196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:01.478915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.480093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:01.480939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:01.483496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 283/559] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:01.484567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.574133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.587664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 277/560] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:01.588679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.600665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9256ae-a930-4821-b0f5-55941fe7b0f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.645829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.647308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.648615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.649213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.649941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.651744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.652265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.654068 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.678510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 284/561] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:01.678510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.705880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f22ef5-ac02-4eb8-b7de-2bfb6e565a48 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.741589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.742318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.742608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.742883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.743643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:01.743956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.745377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40eec1-2d87-4a91-a1cc-c5ad5c77231b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.755424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 278/562] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:01.756748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211920e-ecc5-4a59-b9ed-f2fdd8be9c3b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.757041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211920e-ecc5-4a59-b9ed-f2fdd8be9c3b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.757327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211920e-ecc5-4a59-b9ed-f2fdd8be9c3b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.759365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 279/563] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:01.760097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211920e-ecc5-4a59-b9ed-f2fdd8be9c3b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.786563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7c24-3bd4-41bf-b985-707c4bfc383e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.826525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.827937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.828393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.829005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.832476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.840715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.842233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0021d81b-ad4a-435b-855e-4c415c7822ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:01.859838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.861404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.862423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:01.862423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:01.862423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:01.862423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:01.862423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 285/564] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:01.862961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90327d-33b5-48d0-b335-efcfb9ec8e1d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.863253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90327d-33b5-48d0-b335-efcfb9ec8e1d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.863536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90327d-33b5-48d0-b335-efcfb9ec8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.875435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211920e-ecc5-4a59-b9ed-f2fdd8be9c3b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:01.881249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eb90327d-33b5-48d0-b335-efcfb9ec8e1d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:01.882539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eb90327d-33b5-48d0-b335-efcfb9ec8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:01.885322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:01.911620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:01.912420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.912712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.912993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.923848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:01.925325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:01.942660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:01.943562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:01.944451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:01.944451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:01.944451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:01.944451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:01.944451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 280/565] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:01.945082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:01.952020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 286/566] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:01.953337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d23204-2df1-4242-be69-188e1cb000ef None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:01.953629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d23204-2df1-4242-be69-188e1cb000ef None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:01.954060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d23204-2df1-4242-be69-188e1cb000ef None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:01.958033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 287/567] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:01.958033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d23204-2df1-4242-be69-188e1cb000ef None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.049908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6fc9b-9be1-49cc-b53b-e373e33c42fa None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.067161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d23204-2df1-4242-be69-188e1cb000ef None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.083499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.084232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.084523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.084805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.085182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.095767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.097225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.103010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.103676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.103962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.104243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.104975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.105290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.106695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437813c-6db6-437c-9698-c50a8f6a39b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:02.114391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.115263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.116150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:02.116150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:02.116150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:02.116150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:02.116150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 281/568] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.117066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.119527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 288/569] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:02.120836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.121199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.121552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.132346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.138071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:02.139286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:02.206206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 289/570] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:02.207041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.241364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd84d7a-6279-4f99-b50b-809eafebcc23 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.279626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.279823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.280037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.280258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.281009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.281371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.283603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.296342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 282/571] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:02.297220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.337469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dad319-cac0-43b9-adb7-d414f88e9f81 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.347372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.374777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.375774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.376136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.376495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.380060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.388322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:02.397075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.398309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 290/572] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.403021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.419196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19273d23-df19-4eac-baaf-a8a77f287993 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.464870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.465639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.466043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.466327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.474011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.475451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:02.489523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:02.491398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 283/573] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.492445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.522665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fcd9d-c0e3-4b55-9df1-be5b69c1a398 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.562356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.563310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.563694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.564338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.565152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.565476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.566997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.580066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 291/574] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:02.580918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.606767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd7010-583f-4830-9f4d-efe1ff978814 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.608405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.635428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.641801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.642790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.643082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.643363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.653811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:02.662427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.663425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:02.664271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 284/575] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.664981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.684419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fc9d5-4c4b-453a-b3d3-c9892268263f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.718384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.719183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.719583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.720625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.727298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.728736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:02.742695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.743595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:02.744540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 292/576] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.745028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.763422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916485a3-0cc2-485b-958b-dab703957149 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.798944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.799616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.799903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.800188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.800938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.801257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.802652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.810715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 285/577] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:02.811501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.833281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.845475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28357c3b-54b0-459b-afbb-1bbcabe9ce28 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.868228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:02.879784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.880458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.880750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.881035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.881872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:02.882195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.883637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.897108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 293/578] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:02.897987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.917634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1426c6-2407-47d5-8491-766fffa17030 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:02.951384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:02.952540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:02.952899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:02.953267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:02.961739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:02.963182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:02.977313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:02.978350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 286/579] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:02.985190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:02.997518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4660-a445-426a-a92a-1fa28e04a188 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.034270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.034939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.035229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.035505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.036258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.036577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.037970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.045895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 294/580] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.046835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.059165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.090384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70674194-03cf-4c78-9c94-25255a2a064b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.103862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.126604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.127511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.127958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.128245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.129044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.129365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.130825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.148110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 287/581] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.148110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.153059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0301-c9b3-44cc-9f0f-d1058efa2211 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.186961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.187752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.188043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.188907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.199519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.201064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:03.218284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:03.219310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:03.220273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:03.220273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:03.220273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.220273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:03.220273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 295/582] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:03.220727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.243534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739e457-6e7d-4186-9548-62e6c9f11935 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.278634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.279287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.279576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.279857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.280697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cdf6e3c71de545d782c99ca87d86fb1e) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.281019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.282577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.284065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.288553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 288/583] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/role_assignments?user.id=cdf6e3c71de545d782c99ca87d86fb1e&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.289324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.335828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa48f8-9c23-4d5c-bf8e-aaf21e99a4c5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.344504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.377319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.377915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.378210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.378492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.379697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.380020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.382309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.405862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 296/584] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.405862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.420333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fffe4c-ddeb-41e3-a06e-0deb0ebd5688 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.466023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.466841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.467135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.467425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.478015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.479467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:03.497993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:03.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:03.500013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:03.500013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:03.500013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.500013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:03.500013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 289/585] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:03.500013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.526942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b8349-5342-4944-a126-1511ee93d94c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.549787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.566069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.566936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.567336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.567619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/f5953aa40eb54d399b593000f3c74596/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.584376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=f5953aa40eb54d399b593000f3c74596, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.585939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.600069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 297/586] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:03 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/f5953aa40eb54d399b593000f3c74596/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:03.601166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.604871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.619563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0c0b-8389-4894-8146-dbcf43bf4f3e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.662539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.663508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.663980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.664441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.665420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.666011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.667779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.682695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 290/587] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.684018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.761239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8414a97-2ede-46a9-b219-47a754b0ac7c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.804878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.805725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.806107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.806389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.819310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.820893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.826444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e863-f547-4423-bd5e-437fb7973d84 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:03.844999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:03.846583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:03.847478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:03.847478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:03.847478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:03.847478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:03.847478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 298/588] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:03.848452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.872133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:03.872981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:03.873276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:03.873564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:03.874717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:03.875038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:03.876790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:03.884932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.893028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 291/589] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:03.898394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:03.918146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:03.995315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a8ec2-5e00-492f-9477-15ffe0f38ffa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.029991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f493b71-97ec-4036-aa9c-dec35cfbe395 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.034477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.035302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.035595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.035878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.043731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.045182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:04.061961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.063189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:30:04.065232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:04.068913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 299/590] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.069841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.075372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.076328 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.077070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.077556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.078526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.079926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.081696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.095702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 292/591] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.097209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.161983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.185565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8201-5628-4126-a852-8ba98acccdff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.193005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.224096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.224907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.225200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.225480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.233042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.234526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.238833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eb281-9fc0-47cb-8e8d-192ef82298f3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:04.250449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.252329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:04.253283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:04.256973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 300/592] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.257853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.285684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.287061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.287826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.288253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.290055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.290554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.292982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.317780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 293/593] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.318851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.385056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6b3a4-6dc1-4231-92aa-1f57a91635a1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.423480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.424537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.425092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.425570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.433621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.435627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:04.454009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.455096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:04.456010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:04.457768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fba69-aefa-44d1-9c6a-1547e11c45f8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.458808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 301/594] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.460509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.463736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.491464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.501906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.502848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.503310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.503790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.504808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.505292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.507043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.530863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 294/595] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.532992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.570097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecda5a-2796-4d0c-a256-a87e6cf1c314 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.605858 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.606763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.607144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.607491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.620175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.621859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:04.640250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.641167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.642615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:04.642615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:04.642615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:04.642615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:04.645921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 302/596] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.646755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.648687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2ca7-19d9-40ad-adde-e3389f0e2ae2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.687306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.688346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.689152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.689643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.690773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.691297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.693025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.708038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.708564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 295/597] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.710474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.756308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83913fa-3ba0-465f-9ecf-0a86f3247061 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.763241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.794197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.795667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.795667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.795667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.807276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.808673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:04.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:04.828159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:04.829877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:04.829877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:04.829877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:04.829877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:04.833256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de9ca8-4059-4f9a-903f-817c713638d2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.833393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 303/598] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:04.834226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.872577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.873608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.874115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.874619 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:04.875548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:04.876265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:04.878088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:04.893691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 296/599] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:04.894883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:04.955304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895e93b-ad2d-401e-8c7f-8d11e98c54b9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:04.962592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:04.991506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:04.992420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:04.992710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:04.993197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/3ca2cd515aa84a3ca01656977036ac07/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.009216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=3ca2cd515aa84a3ca01656977036ac07, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.010769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.012126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835b1d-4e43-463f-a4e5-c5c10af53804 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.022903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.033819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 304/600] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:04 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/3ca2cd515aa84a3ca01656977036ac07/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:05.034852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.056022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.057076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.057812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.058290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.059229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.059745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.061366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.075762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 297/601] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:05.086794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.247706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2c9d6-9618-4ace-821e-f7051df437a3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.265729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ad299-dd19-4f43-b1ec-2e4237a987f1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.287061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.287936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.288165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.288444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.295630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.297095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.308756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.310170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.310638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.311113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:05.311728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:05.312587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:05.313560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 305/602] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:05.318913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.321926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.323650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.338350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:05.341949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:05.343032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:05.344045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:05.348026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 298/603] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:05.355080 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.420911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.468606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860e17-ba4c-4780-b40a-8b61b8e3f5a5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.506507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.507416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.507711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.508270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.509084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.509406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.511444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7377eb-cf78-4e68-b511-8588d1c304e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.528591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 306/604] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:05.531103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4546ffa-a962-4711-b3a8-904345abd90a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.531393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4546ffa-a962-4711-b3a8-904345abd90a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.531670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4546ffa-a962-4711-b3a8-904345abd90a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.535010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 307/605] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:05.536359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4546ffa-a962-4711-b3a8-904345abd90a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.538603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338b94-456d-4acc-8c7e-5c488b03c11a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.575339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.576037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.576379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.576661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.577805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:05.578124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.580464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638753c6-cca6-4151-a443-d5a24b20c28e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.593472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 299/606] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:05.595532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.596046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.596434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.613369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:05.614609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:05.634731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.667588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4546ffa-a962-4711-b3a8-904345abd90a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.692584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 300/607] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:05.693467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.709908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.710963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.711255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.711537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/d292189b741b40b8859bfe9883b4c5ec/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.715573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:05.730378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=d292189b741b40b8859bfe9883b4c5ec, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.732413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.762372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 308/608] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:05 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/d292189b741b40b8859bfe9883b4c5ec/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:05.763151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:05.826440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2226db-5bd8-4ad7-9efd-6b7fba305c9b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:05.873411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601c02a-1072-480d-9e4e-3b8e329f3093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:05.874649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601c02a-1072-480d-9e4e-3b8e329f3093 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:05.874929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601c02a-1072-480d-9e4e-3b8e329f3093 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:05.875206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601c02a-1072-480d-9e4e-3b8e329f3093 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/a26d10743fe844a98ecfcae3974dbce4/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:05.896687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601c02a-1072-480d-9e4e-3b8e329f3093 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=a26d10743fe844a98ecfcae3974dbce4, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:05.898269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601c02a-1072-480d-9e4e-3b8e329f3093 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:05.916757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 301/609] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:05 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/a26d10743fe844a98ecfcae3974dbce4/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:05.977696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d64f0a-0724-4b7f-9cb8-495a66e1b9d0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.009886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.021379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec600ec-3d5a-4954-aea9-2b342bb03495 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.021694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec600ec-3d5a-4954-aea9-2b342bb03495 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.021900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec600ec-3d5a-4954-aea9-2b342bb03495 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.023572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 302/610] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:06.029779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16409-115b-4acc-af17-42a851741e5a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.029973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.030606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16409-115b-4acc-af17-42a851741e5a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.030785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.031078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.031417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.031695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16409-115b-4acc-af17-42a851741e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.048097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:06.054096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-11a16409-115b-4acc-af17-42a851741e5a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:06.055683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-11a16409-115b-4acc-af17-42a851741e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:06.057091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:06.058188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:06.059055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 309/611] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:06.065847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.146050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.158960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 303/612] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:06.184590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16409-115b-4acc-af17-42a851741e5a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.242883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dbc87-7b47-4460-8ac0-d64992eab03e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.304269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.304963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.305280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.305568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.306364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:06.306686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.308071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.321311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 310/613] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:06.327652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.348472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16409-115b-4acc-af17-42a851741e5a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.409164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.427289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.428505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.429064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.429557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.454364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:06.462248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:06.463200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:06.464053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:06.470128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 304/614] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:06.477014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.478343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a46bc-eb92-4382-9aee-ac545dce8afb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.548213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.549518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.549835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.550116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.565329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.566967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.587918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:06.595123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:06.596140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:06.597066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:06.597066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:06.597066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:06.597066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:06.597066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 311/615] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:06.603886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.640539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb354b6c-8f73-4ed7-a382-49a2b3657e01 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.678440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.679178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.679473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.679759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.680576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:06.680898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.682326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.691508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 305/616] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:06.698626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.780311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.786170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37b154-a426-495a-ab26-c36cc6bbc7ca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.854827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.855572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.855996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.856298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.857044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:06.857362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.859256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:06.881776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125306-53fb-4adb-b09c-13242f3b8b66 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:06.886378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 312/617] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:06.893244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:06.916813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:06.952675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:06.953903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:06.955546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:06.956040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:06.980235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:06.984011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:07.021449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:07.022424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:07.023639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:07.023639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:07.023639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:07.023639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:07.027251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 306/618] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:07.033948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.089679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef1655-62b1-42c5-9e05-48fbdb2ed346 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.131489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.132636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.133148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.133714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.141789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.143494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:07.159539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:07.160538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:07.161690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:07.164397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 313/619] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:07.173263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.223685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.236967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecb347-c15c-49bc-bbf3-fec4abd55520 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.277379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.278111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.278397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.278679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.279418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:07.279733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.281146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.301500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 307/620] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:07.303021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.358034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.405505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e2c0-0e12-42b8-a405-fb3e4f1a24c9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.441262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.442092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.442381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.442657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.452380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0c78-0bea-45d9-a6e9-63d2c9f22f23 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.458477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.458477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:07.473621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:07.474607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:07.475580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:07.479162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 308/621] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:07.480525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.529085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.529807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.530102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.530385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.531251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:07.531569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.533625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.568295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 314/622] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:07.582682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.599211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521ac2e-aa52-4cb4-ada1-f6a272dc2208 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.601995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.645428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.646270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.646657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.647032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.647998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:07.648412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.649961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.673683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 309/623] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:07.676311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:07.791843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbbdcd-7876-4fc1-a8ab-47285fc18575 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.833192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.838927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f1eb8-0b0b-4b7c-b452-d706d7ff12fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.841710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1eb8-0b0b-4b7c-b452-d706d7ff12fd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.841710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1eb8-0b0b-4b7c-b452-d706d7ff12fd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.841710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1eb8-0b0b-4b7c-b452-d706d7ff12fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.872831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:07.914679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50397405-ebba-4990-9cf8-d44fd49d1e28 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:07.931763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f1eb8-0b0b-4b7c-b452-d706d7ff12fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.933993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f1eb8-0b0b-4b7c-b452-d706d7ff12fd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:07.974610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:07.976424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:07.976910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:07.977379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:07.993493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:07.995787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0124c600-d865-4fd5-9f95-466b6e809732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:08.022091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:08.023115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:08.024066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:08.024066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:08.024066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:08.024066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:08.027000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 315/624] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:08.051249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4c9cb-6cf6-461d-8459-21c07cd3219d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.051557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4c9cb-6cf6-461d-8459-21c07cd3219d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.051824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4c9cb-6cf6-461d-8459-21c07cd3219d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.056624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 316/625] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:08 2022] GET /identity => generated 270 bytes in 29 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:08.059119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4c9cb-6cf6-461d-8459-21c07cd3219d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.098634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 310/626] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:30:08.100435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.100726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.101008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.133144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:08.134363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:08.217498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:08.264873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 311/627] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:08.266216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.277305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4c9cb-6cf6-461d-8459-21c07cd3219d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.324951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:08.335263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.336396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.336856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.337326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.338404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:08.339084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:08.341585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:08.357007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 317/628] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:08.358304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.442702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d95e7c-a027-47e1-94b7-907fcb513e17 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.485974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80832b-c756-45ce-8d3f-28c756314cae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.486264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.487308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.487676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.488063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.500207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:08.501894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:08.519300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe907c0-97be-4dee-98ea-97babd7c472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:08.520038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:08.521155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:08.522119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:08.522119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:08.522119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:08.522119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:08.522364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe907c0-97be-4dee-98ea-97babd7c472e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.522364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe907c0-97be-4dee-98ea-97babd7c472e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.522364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe907c0-97be-4dee-98ea-97babd7c472e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.522560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 312/629] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:08.522560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.536069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe907c0-97be-4dee-98ea-97babd7c472e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4fe907c0-97be-4dee-98ea-97babd7c472e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:08.543928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:08.544996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:08.547345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 318/630] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:08.548244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1a972-0754-40f9-9ec2-ef68baf8ef92 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.549795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1a972-0754-40f9-9ec2-ef68baf8ef92 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.549795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1a972-0754-40f9-9ec2-ef68baf8ef92 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.551234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 319/631] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:08.552210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05887b25-aaae-4c95-af78-f5803633c982 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.552498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05887b25-aaae-4c95-af78-f5803633c982 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.552773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05887b25-aaae-4c95-af78-f5803633c982 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.554029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 320/632] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:08.555062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05887b25-aaae-4c95-af78-f5803633c982 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.559278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:08.609565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:08.631199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea0690-47c2-493c-8946-656d4ce51921 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.663242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05887b25-aaae-4c95-af78-f5803633c982 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.665654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.666482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.666766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.667043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.674169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:08.675611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:08.689187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:08.690254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 313/633] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:08.691165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.699315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.699979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.700264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.700539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.701263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:08.701574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:08.703213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ac8d7-fded-406c-a935-e2cbc59de8a3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:08.715384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 321/634] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:08.716591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159d9d-c6d1-4663-b820-761c3c013c49 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.716883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159d9d-c6d1-4663-b820-761c3c013c49 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.717158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159d9d-c6d1-4663-b820-761c3c013c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.733582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e9159d9d-c6d1-4663-b820-761c3c013c49 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:08.734788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e9159d9d-c6d1-4663-b820-761c3c013c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:08.794142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:08.801055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 322/635] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:08.802389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.802687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.803000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.805053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18f872-7d8b-4cda-b673-25f1500c4b4f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.820086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:08.821510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:08.841265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.841989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.842280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.842562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.843320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:08.843635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:08.845150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:08.853245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 314/636] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:08.854103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.858513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:08.892471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 323/637] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:08.893328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:08.959017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650d986-94d8-4fa9-9881-bcef73d1d209 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.992258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:08.992924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:08.993025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6d9e6-8455-4738-b09a-799683d6b59f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:08.993215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:08.993496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:08.994332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:08.994654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:08.996142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.008707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 315/638] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.009511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.024461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.030581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.031386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.031679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.032079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.038860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.040326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:09.054194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.055136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:09.056135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 324/639] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:09.056650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.093143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.115994 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870468d4-0f2b-4571-a173-74ab3d01bca1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.150312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.151114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.151403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.151686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.162116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:09.164205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684e481-81aa-44be-ba5a-fda043285864 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:09.169886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.170860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:09.171697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 316/640] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:09.173912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.203115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.204048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.204547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.204642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.215595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.217055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:09.236589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.237820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.238931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:09.238931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:09.238931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:09.238931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:09.238931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 325/641] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:09.238931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.250290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.280934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad7d11-1bfe-4d96-8aa8-467a23e47f54 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.318060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.318891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.319185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.319469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.322557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.330146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:09.337962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:09.339089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 317/642] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:09.340479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.347731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea63299-b00e-4883-a7da-a06c3d5c59d2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.382047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.382946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.383234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.383510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cdf6e3c71de545d782c99ca87d86fb1e/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.398163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=cdf6e3c71de545d782c99ca87d86fb1e, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.399619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.413426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 326/643] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:09 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cdf6e3c71de545d782c99ca87d86fb1e/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:09.414279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.451299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856cc8e-3d51-4414-84f8-246e1d108ca3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.489604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.498308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.499084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.499444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.499795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.502127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.502462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.504617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.521021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 318/644] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.522097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.562999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b42731-2700-4da4-9f04-b2de19ebb446 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.602600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.610673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.611870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.612166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.612483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.615876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.619327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.620747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.631378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 327/645] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.632360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.652348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995430be-1974-4f7e-bc22-0cb759a7edf9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.693573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.694287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.694576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.694855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.695587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.695901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.697279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.713405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 319/646] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.713405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.744276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9c85b-1e0c-43a1-9c6a-7147006656fa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.768046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.780979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.781716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.782529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.783046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.784011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.784332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.785729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.798366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 328/647] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.798366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.824838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324f58-f250-4d35-abcb-36ab23442cdd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.861887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:09.864059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.864732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.865106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.865451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:09.866337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=f5953aa40eb54d399b593000f3c74596) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:09.866652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:09.868079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:09.881707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 320/648] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/role_assignments?user.id=f5953aa40eb54d399b593000f3c74596&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:09.881707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:09.939739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80387a4c-81ca-46a7-a7f7-216bdb5b0d9f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:09.987813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:09.989257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:09.989564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:09.989892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.005472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.007522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.026063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e0d74-43f2-4e6f-9e6a-b8f79a3b7627 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:10.030873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.031810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.032710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:10.032710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:10.032710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:10.032710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:10.037476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 329/649] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.038510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.056434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.084413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.085340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.085630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.085953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.099542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.101003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:10.121920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 321/650] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.124289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.186589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f90d0a-0023-4add-b74f-1ac5332c8f7f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.207916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.245012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.246095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.246253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fc4fd-0b10-4247-bdde-09389a48b108 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.246696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.247120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.262222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.264618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.287844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.288514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.288979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.289270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:10.289998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.290883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.291779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:10.291779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:10.291779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:10.291779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:10.292023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.292023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.292023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.294185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 330/651] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.295560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.306059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 322/652] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.307190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.345742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.428789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e10315-cdee-4efa-b30a-f262417849ce None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.429117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9005f-4960-492e-9aa7-472b545e61d4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.474232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.475094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.475482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.475873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.476828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.477066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.477335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.478157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.478675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.479121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.480108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.480594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.480948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.483693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.498753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 323/653] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.501703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.507163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 331/654] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.507327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.535044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.614078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.621413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b34ec1-c5df-401c-8335-9db533c31085 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.646727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11454d-9d13-4ae8-a2ab-acdbee3c40ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.658246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.659013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.659405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.659788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.668298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.669935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:10.686244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.687410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:30:10.688614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:10.689947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.690578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 324/655] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.691226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.691655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.691774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.692268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.701772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.703663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:10.721150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:10.722235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:10.723174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:10.726091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 332/656] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:10.727195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.806178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966167-7f97-4a29-97ae-db1f52d003a5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.815040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.841412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b26f7-4a2a-4100-ad33-12bd6f16cb5d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:10.852707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.854061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.854959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.855426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.882522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:10.883329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:10.883685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:10.884037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:10.884959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:10.885373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.887561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:10.889151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:10.914043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 333/657] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:10.915341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:10.946516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:10.948020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.034207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6674b34-8276-43f4-86c6-9b513220a690 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.062909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 325/658] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:11.064583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.072077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.073001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.073489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.074103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.075153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.075653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.077617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.081340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.107737 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 334/659] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Jun 15 14:30:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:11.108606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.137898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.185514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd805f4-732c-49ea-ab48-9258b37da4e9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.225542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.226297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.226616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.226899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.242345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6cce9-e926-41af-99f5-2818a7da59f3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.280708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.281957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.282450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.282943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.294387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.296174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.308425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.309901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:11.316760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:11.317642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:11.318586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:11.318586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:11.318586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:11.318586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:11.320883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 335/660] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:11.322021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.372765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.383011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.432217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697b7b7-4487-4e31-85a5-98816c653654 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.437240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 326/661] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:11.437240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.477873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a376b9c-10ee-4e73-a75e-c68fc08a3d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.478947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a376b9c-10ee-4e73-a75e-c68fc08a3d54 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.479482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a376b9c-10ee-4e73-a75e-c68fc08a3d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.479974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a376b9c-10ee-4e73-a75e-c68fc08a3d54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.557519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a376b9c-10ee-4e73-a75e-c68fc08a3d54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.558570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20bb06-2032-48ca-8762-64f818aba8ea None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.560006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a376b9c-10ee-4e73-a75e-c68fc08a3d54 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.601214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.601923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.602215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.602499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.603243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:11.603894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.604921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.611908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.619045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 327/662] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:11.619855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.640714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.678527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 336/663] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:11.679982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18509f0d-cf3b-4b04-9f7b-3441542b4faa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.680283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18509f0d-cf3b-4b04-9f7b-3441542b4faa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.680570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18509f0d-cf3b-4b04-9f7b-3441542b4faa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.682044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 337/664] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:11.682667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18509f0d-cf3b-4b04-9f7b-3441542b4faa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.735889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7671846-06f7-4341-9eaa-25a77ea89bb2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.771317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.772117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.772406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.772685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.787087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.789970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.799895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18509f0d-cf3b-4b04-9f7b-3441542b4faa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:11.816132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:11.817269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:11.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:11.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:11.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:11.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:11.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 328/665] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:11.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.845465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.846368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.846664 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.847043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.854328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:11.855714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-94bfb277-9a21-4110-ad99-e02568a32a81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:11.871345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:11.872263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:11.873147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 338/666] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:11.874266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.874266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.874443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:11.885773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.891418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:11.892631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:11.905403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:11.944548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e4125-3eff-4776-86f9-1366c8b66908 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:11.963725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 339/667] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:11.964764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:11.994866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:11.996128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:11.996736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:11.997475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.009822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.011351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:12.033697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:12.034748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:12.035719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:12.035719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:12.035719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:12.035719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:12.035719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 329/668] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:12.041699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.096361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a117e3d-21d2-4d50-bbf9-40735576d3f2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.138364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.139062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.139845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.139845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.140383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.140701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.142598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.145764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.162498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 340/669] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.163538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.169861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a04ef4-e0d0-4928-9ec8-4bf85e642380 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.193589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.210888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.211588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.211870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.212163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.212960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.213273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.214725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.227512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 330/670] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.228338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.272480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c6548-ded4-4b4a-806a-61529fe04c93 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.305773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.306573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.306858 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.307134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.317802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:12.325137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d02b5-c389-4db7-90be-462ebc93d911 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:12.325561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:12.326540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:12.327638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 341/671] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:12.330087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.364474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.365140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.365435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.365757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.366490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.366807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.368171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937350b-f167-4be6-9437-47010060aac5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.383987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 331/672] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.383987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734791-ff71-4f1b-9be9-dafd58f98322 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.383987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734791-ff71-4f1b-9be9-dafd58f98322 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.383987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734791-ff71-4f1b-9be9-dafd58f98322 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.383987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 332/673] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:12.383987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734791-ff71-4f1b-9be9-dafd58f98322 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.389316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.427936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.435358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549c22d-ea94-4cda-9e16-05a0aedf5e06 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.471808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.472698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.472989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.473273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.479934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.481374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:12.495809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:12.496878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:12.497871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:12.498304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734791-ff71-4f1b-9be9-dafd58f98322 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.498379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 342/674] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:12.498379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.535404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.536060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.536351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.536635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.537479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=3ca2cd515aa84a3ca01656977036ac07) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.537832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.539276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.545166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 333/675] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/role_assignments?user.id=3ca2cd515aa84a3ca01656977036ac07&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.546229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.608632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7f88-39bd-4429-b382-deb34cff4bf3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.629042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.650711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.651591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.651875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.652150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.652917 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.653228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.654654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04019b-77be-4057-a993-91e119ef9d54 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.669635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 343/676] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.669635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.669635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.669635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.673355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33683-7a0a-496f-897b-b23badfee9ec None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.676886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.685919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:12.687136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:12.716468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.717799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.717799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.718000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.729522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.731855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:12.748894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:12.749950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:12.751108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:12.753500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 334/677] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:12.754560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.793813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 344/678] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:12.794550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.886296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2ba5e-544a-4996-a370-0137c16d5c1d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.912349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a50be-395e-47aa-8bfd-8d41fba5ceeb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:12.923764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:12.932787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.933460 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.933845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.934087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.934905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:12.935195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.936600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.949224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 335/679] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:12.950117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:12.958743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:12.959678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:12.960018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:12.960301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:12.972649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:12.974311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:12.977290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:13.003691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:13.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:13.005764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:13.005764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:13.005764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:13.005764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:13.007742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 345/680] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:13.008328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.108467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5951d-0e99-433a-80bc-46e32603cfe2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.164170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d096-33f0-4499-8914-d9e0f2cb6ff2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.172789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.173472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.173782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.174065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.174900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.175219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.176875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.191032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 336/681] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.191822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.226898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.228086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.228547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.229400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.244810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:13.254318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:13.255239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:13.256156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:13.258911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 346/682] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:13.260189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.296210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.321555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.345171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621099b8-4676-4eb7-83f5-50df6f1056e6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.383863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.384472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.384740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.385032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.386010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=d292189b741b40b8859bfe9883b4c5ec) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.386335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.387881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.397812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 337/683] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/role_assignments?user.id=d292189b741b40b8859bfe9883b4c5ec&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.397812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.409828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a0cd4-fdd6-4ae4-9b61-626930e97d6d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.463846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.465573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.466135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.466616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.467708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.468245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.470234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.524401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 347/684] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.525705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.574376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a92f0-c705-45ae-b63d-ab4bd35e628a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.620842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.621500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.621865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.622145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.622993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=a26d10743fe844a98ecfcae3974dbce4) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.623469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.627107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 338/685] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/role_assignments?user.id=a26d10743fe844a98ecfcae3974dbce4&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.636115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.664845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.714057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.734109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22385c-6b0d-4c61-9256-7b75c0067da4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.775518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08051a-abab-4193-b27a-2af0f5f63cce None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:13.785320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.786464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.786972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.787499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.788635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:13.789157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.790898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.803282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 348/686] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:13.811050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.821267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:13.821952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:13.822243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:13.822530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:13.831839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:13.833378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:13.850239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:13.851279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:13.852263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 339/687] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:13.857519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:13.961526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:13.996182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a865f-7b18-4731-9fff-05bb86156cf6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.039073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607bf74-9902-4e5e-8ead-5a56d0f919ae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.058389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.059428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.059936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.060416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.061474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:14.062044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.069703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.089228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.089228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.089228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.089566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.111129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.112618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.114612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 340/688] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:14.124514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.132589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:14.147516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:14.148471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:14.149385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:14.149385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:14.149385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:14.149385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:14.154515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 349/689] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:14.166266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.361690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d42b9-6b3e-4aff-9bde-0f0deafec3d5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.409691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8590035-9484-4cd7-91fe-156668365037 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:14.462400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:14.473355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a618a2-eae5-4bae-9827-ddf4f564e80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.475514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a618a2-eae5-4bae-9827-ddf4f564e80b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.475514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a618a2-eae5-4bae-9827-ddf4f564e80b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.475514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a618a2-eae5-4bae-9827-ddf4f564e80b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.548136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:14.550910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:14.552260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:14.553688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:14.557516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:14.558999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.569322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.601686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 350/690] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:14.609687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:14.697068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a618a2-eae5-4bae-9827-ddf4f564e80b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:14.701694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a618a2-eae5-4bae-9827-ddf4f564e80b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:14.864382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.022920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 341/691] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 899 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:30:15.025092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedfe2f-b455-48b2-80ac-d3f0e572edfb None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.025714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedfe2f-b455-48b2-80ac-d3f0e572edfb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.026606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedfe2f-b455-48b2-80ac-d3f0e572edfb None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.033074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2531-d416-4491-9ff3-4b904cee598b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.033334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 342/692] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:15.034528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedfe2f-b455-48b2-80ac-d3f0e572edfb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.067939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.074147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.074960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.075251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.075530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.083676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.085135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-823da0c5-b7e8-45b7-8da4-f35e27f3cead None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:15.100966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:15.102071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:15.103017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 351/693] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:15.104107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.104204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.104413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.121817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:15.123023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:15.160647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedfe2f-b455-48b2-80ac-d3f0e572edfb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.199401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.200677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.201168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.201701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.213699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 352/694] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:15.213699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.227118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.229948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.260796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:15.268112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:15.269039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:15.270403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:15.270403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:15.270403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:15.270403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:15.272543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 343/695] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:15.273652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.320824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51062-0bc6-44fd-99d4-444a126a775a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.356474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.357197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.357491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.357800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.358658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:15.358974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.361028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.368511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.379930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 353/696] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:15.380828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.389996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f81c-c3e6-4356-8684-6f0a04f87535 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.427973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.429146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.429635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.430262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.442398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.444148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:15.466444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:15.467412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:15.468315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:15.468315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:15.468315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:15.468315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:15.475417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 344/697] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:15.476537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.491156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51b791-3374-411f-95a3-f77a7aedc4fa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.492743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.531423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.532988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.533173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.533446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.534211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:15.534527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.535961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.552656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 354/698] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:15.553557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.591358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366fd31-3c99-4436-b5a3-10e4958aaaf0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.630144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.631180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.631674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.632215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.638060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.665065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b84f79-c138-49c0-ad71-d70f609e3b65 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.700608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.701244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.701534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.701878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.702867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:15.703183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.705163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.713376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.715259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.724639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 355/699] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:15.727885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.761279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.841741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 345/700] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:15.843363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.849117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758414-cbc5-4060-a502-91053dcad9f9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:15.886870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:15.889187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:15.889405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:15.889685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:15.890043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:15.899058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:15.900517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:15.922649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:15.924704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:15.925620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:15.932770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 356/701] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:15.934542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:15.980268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f954f4-c668-4243-8a10-15b739094bdf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.034152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.035337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.035635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.035923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.036678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.038813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.049254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.078133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 346/702] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:30:15 2022] POST /identity/v3/users => generated 254 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:16.079768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.081335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15451b8c-c3ee-4e44-bdbd-02f1dc3dd3d4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.124057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.126009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.126766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.127352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.143822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.146082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4ee52a80-78f5-41c8-a93c-c98e450d2dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:16.175056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.176036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.177029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:16.177029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:16.177029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:16.177029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:16.179540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 357/703] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:16.183199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e29d5-384b-473f-9d22-c9d5376fe898 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.183732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e29d5-384b-473f-9d22-c9d5376fe898 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.184361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e29d5-384b-473f-9d22-c9d5376fe898 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.186545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 358/704] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:16.187827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e29d5-384b-473f-9d22-c9d5376fe898 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.234160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.245979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8c052-dcdc-4a84-9f1d-f34401ec6904 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.303065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.304097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.304601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.304988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.305992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.306441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.308259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c71101-cbcb-4766-92ca-ee877c606f89 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.322704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 347/705] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.323130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.323421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.323727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.342779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:16.344112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:16.345862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e29d5-384b-473f-9d22-c9d5376fe898 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.393304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.394088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.394466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.394926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.396099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.396782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.398584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14fa0-feb8-49ff-b454-4e5f15680ec5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.401121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.413754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 359/706] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.415322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42443f-3e20-4736-a941-83e13ed1de63 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.415811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42443f-3e20-4736-a941-83e13ed1de63 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.416313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42443f-3e20-4736-a941-83e13ed1de63 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.418973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 360/707] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:16.420257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42443f-3e20-4736-a941-83e13ed1de63 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.429930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 348/708] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:16.431045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.533734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.545743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42443f-3e20-4736-a941-83e13ed1de63 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.555135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de3f2-dfe7-4a17-9fe2-a9426909285a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.584941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.585749 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.586108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.586482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.587375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cdf6e3c71de545d782c99ca87d86fb1e) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.587770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.589359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865a54d-a5fd-45ae-86cc-398070ad60b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.594543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.595421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.595776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.596126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.606096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 361/709] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/role_assignments?user.id=cdf6e3c71de545d782c99ca87d86fb1e&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:16.606096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.606096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.606096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.618999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.621082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.641780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:16.641780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:16.647595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.648579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:16.649492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:16.650758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 349/710] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:16.651793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.743312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.773725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 362/711] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:16.773725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.807540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef79e1d-24d0-46f9-9305-3ee90c1f105a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.859516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.861241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.861926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.862432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] PATH_INFO: `/v3/services/network` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.863442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.865293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:16.884122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:16.885289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:16.886279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:16.887636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 350/712] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/services/network => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:16.889837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:16.893351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:16.926962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11812f-9bde-4832-9ff1-c284a8693cb6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:16.978020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:16.979197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:16.980203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:16.980670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:16.981990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:16.982633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:16.985949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.003410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 363/713] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.005178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.020902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c887e-fdb9-4ffa-a786-9ac51d33ed50 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.066940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.071050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.073434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.074138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.074688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.075919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.079322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:17.096480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:17.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:17.098704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:17.101942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 351/714] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Jun 15 14:30:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:17.102931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.142406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d72ba8-dc60-4d6a-8058-3bff0c64fc4b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.189013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.201496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.202779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.203256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.204690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.205613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] RBAC: Adding query filter params (name=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.206578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.208455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.215400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 364/715] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.216406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.237042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a0c09-bfb6-4c38-b02f-4b732d360537 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.283953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.284877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.285234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.285789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.286795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=f5953aa40eb54d399b593000f3c74596) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.287191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.289214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.301835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 352/716] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/role_assignments?user.id=f5953aa40eb54d399b593000f3c74596&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.303908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.395880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82005f5-37be-4475-bdfc-8500acf00538 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.430045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.494440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.495480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.495951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.496400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.497307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.497830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.499394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.505685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494ecb-5f85-4462-a621-023b59254842 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.513684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 365/717] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.521685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:17.539922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.540610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.540903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.541191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.541979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] RBAC: Adding query filter params (type=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.542476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:17.543331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.546172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be223-856e-4f1e-83cb-b8f9b096dded None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.557810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 353/718] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:17.733861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11bd12-e14b-49f1-941f-3521c3d90c06 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:17.769585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:17.770409 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:17.770702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:17.770980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:17.771695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:17.772003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:17.773360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e81d-a1c6-4c8b-9b12-4171058c858e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:17.779336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 366/719] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:18.517106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa791ab9-699a-486a-8fb6-91ddac56a2ed None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:18.517617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa791ab9-699a-486a-8fb6-91ddac56a2ed None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:18.517717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa791ab9-699a-486a-8fb6-91ddac56a2ed None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:18.519482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 354/720] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:18.524978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:18.525273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:18.525556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:18.554686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:18.556246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:18.629994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:18.727373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 367/721] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:18.775992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa791ab9-699a-486a-8fb6-91ddac56a2ed None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:18.956436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa791ab9-699a-486a-8fb6-91ddac56a2ed None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.026520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9664dd4-643b-4b32-8038-07705ffaddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.027404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dd4-643b-4b32-8038-07705ffaddaf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.027700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dd4-643b-4b32-8038-07705ffaddaf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.027981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dd4-643b-4b32-8038-07705ffaddaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.045221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9664dd4-643b-4b32-8038-07705ffaddaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b9664dd4-643b-4b32-8038-07705ffaddaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:19.053247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:19.054284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:19.055157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 355/722] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:19.062085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.284274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c18962-8087-4b34-8f8f-9f8754567627 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.299128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba861a9-f67e-458e-a91b-136e44be0c8f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.299439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba861a9-f67e-458e-a91b-136e44be0c8f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.299704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba861a9-f67e-458e-a91b-136e44be0c8f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.313711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 356/723] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:19 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:19.324290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7950601-8942-470e-aba6-68049636062e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.324290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7950601-8942-470e-aba6-68049636062e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.324290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7950601-8942-470e-aba6-68049636062e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.329440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.331134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.331582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.333166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.335550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:19.336265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.338540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.356724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 368/724] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:19.358958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.359246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b7950601-8942-470e-aba6-68049636062e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:19.360648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b7950601-8942-470e-aba6-68049636062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:19.396848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.477559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855ecd7-4235-4b3a-b520-b1ec7a38262c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.514102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.515394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.517934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.521653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.522200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.539797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.541571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.559691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 357/725] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:19.569291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:19.570367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:19.571280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:19.571280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:19.571280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:19.571280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:19.573551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 369/726] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:19.582759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7950601-8942-470e-aba6-68049636062e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.592519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.762043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.777706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7950601-8942-470e-aba6-68049636062e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.805162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a581e4-611c-492f-9528-518bd39d1ffb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:19.844341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.845726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.846024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.846309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.847058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:19.847373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:19.848756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b931-3a50-4876-93d4-be03b61347db None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:19.868269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 358/727] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:19.869710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb6546-6b3c-4684-8596-7204d78b8a20 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.870008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb6546-6b3c-4684-8596-7204d78b8a20 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.870293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb6546-6b3c-4684-8596-7204d78b8a20 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.872220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a78dd-3c0c-418b-aea0-5beaf0061031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:19.873324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a78dd-3c0c-418b-aea0-5beaf0061031 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.873886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a78dd-3c0c-418b-aea0-5beaf0061031 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.874367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a78dd-3c0c-418b-aea0-5beaf0061031 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.876602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 359/728] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:19.877801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb6546-6b3c-4684-8596-7204d78b8a20 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:19.888624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a78dd-3c0c-418b-aea0-5beaf0061031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a59a78dd-3c0c-418b-aea0-5beaf0061031 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:19.899789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:19.900767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:19.901706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:19.904548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 370/729] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:19.906958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:19.907808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:19.908247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:19.929536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:19.931307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:19.933998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:19.999566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb6546-6b3c-4684-8596-7204d78b8a20 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.013904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 371/730] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:20.014813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.038463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.039752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.040305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.040831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.050242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.052500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:20.072424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.073395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:20.074377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:20.077451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 360/731] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.079495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.115529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.141252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bf325-651d-4156-aa45-df17a75c02ea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.179616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.180171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.180467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.180749 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.181513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.181895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.183241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.196214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f456ea-d216-41c3-b1bb-63bb1b7e61a0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.198769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 372/732] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.199986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.205810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.241384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.242701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.243225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.243705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.257042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:20.266352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.267317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:20.268170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:20.271035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 361/733] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.272166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.313191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae31a06-b45c-4bee-9447-75d13b1948f7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.349628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.350267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.350558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.350841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.351702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.352004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.353429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.356135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.372622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 373/734] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.373532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.395521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a5f8c-e38b-4c89-a587-457da0c0c231 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.435927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.436724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.437012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.437293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.455013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.456491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-356f1d60-2054-4124-8714-e440b6a7ed39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:20.474819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.475941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.477089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:20.477089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:20.477089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.477089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:20.477089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 362/735] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.477718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d1959-b7bf-4132-b8b2-345b7a35ef65 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.477718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d1959-b7bf-4132-b8b2-345b7a35ef65 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.477850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d1959-b7bf-4132-b8b2-345b7a35ef65 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.481797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 363/736] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:20.482570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d1959-b7bf-4132-b8b2-345b7a35ef65 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.492204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2290a11-6982-42f1-b117-20df7bd4da95 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.501545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.527286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.527966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.528260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.528543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.529297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.529614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.531015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794dbe-dfe7-4689-ac4c-1addf589c085 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.539871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 374/737] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.541189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b67c-97b4-42b2-8140-594accca6771 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.541480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b67c-97b4-42b2-8140-594accca6771 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.541794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b67c-97b4-42b2-8140-594accca6771 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.543175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 375/738] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:20.543792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b67c-97b4-42b2-8140-594accca6771 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.589514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d1959-b7bf-4132-b8b2-345b7a35ef65 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.603377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.624249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f8bad-0eba-4b02-87d4-e073bc396063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.624909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f8bad-0eba-4b02-87d4-e073bc396063 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.625197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f8bad-0eba-4b02-87d4-e073bc396063 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.626155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f8bad-0eba-4b02-87d4-e073bc396063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.650784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b67c-97b4-42b2-8140-594accca6771 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.684465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.685142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.685440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.685758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.686491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:20.686808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.688306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947c735-48d2-48fa-b958-434dd04d859b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.692828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f8bad-0eba-4b02-87d4-e073bc396063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.694297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f8bad-0eba-4b02-87d4-e073bc396063 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.700865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 376/739] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:20.702389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.702681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.702960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.716383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.718720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:20.720666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:20.788430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 377/740] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:20.789361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.801891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 364/741] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:30:20.803559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.804135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.822452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:20.823652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:20.840405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.893269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 365/742] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:30:20.893992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.894425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e6cd9-0742-41cd-abe2-16b769667b55 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:20.931540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:20.932345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:20.932662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:20.933098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:20.944748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:20.944902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:20.946270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:20.964186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:20.965276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:20.966348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:20.966348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:20.966348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:20.966348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:20.970019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 378/743] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:20.970874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:20.998924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf5273-8475-45e4-a56b-b11808c25dbb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.034033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.034828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.035116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.035396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.042770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.044229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:21.059275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:21.060220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:21.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 366/744] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:21.061789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.071827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.075964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973325-675d-40d4-a200-f576bfd0a917 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.110735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.111534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.111821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.112122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.122500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.123949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:21.142375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:21.143532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:21.144840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:21.144840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:21.144840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:21.144840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:21.144840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 379/745] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:21.144840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.166192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1778b-ba40-4a9f-a08a-86395985496f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.176217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.204715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.205384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.205698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.205976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.206709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.208069 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.223198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 367/746] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/services => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:21.224039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.260750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5e55d-e11f-4303-aa0d-c4fbaff86892 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.307039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.307730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.308019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.308297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.309049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.310490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.329106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 380/747] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:21 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:21.330989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.344387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.368116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14306f-0a2c-41f6-bb9a-f0be9b6dfdfe None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.441376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.442342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.442804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.443088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.445731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.445731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.445731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.467169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 368/748] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.468341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.499063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d60bc-9f28-4a18-a4b8-233d3ee0f1e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.520929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.546695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.547515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.547814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.548087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.549098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.549417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.550946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.575462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 381/749] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.576593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.628269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654750e8-3d3f-4c75-89a7-2d951d3b1a78 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.655907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.676438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.677210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.677589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.678251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.679528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:21.679917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.681626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.699423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76503b-b6e4-4d2b-8b8a-6b9822882c91 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.705325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 369/750] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:21.705693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.746572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.747984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.748511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.749015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.759547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.761859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:21.782276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:21.783298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:21.784237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:21.791813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 382/751] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:21.791813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:21.814429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.840443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155f1c0-9e80-4d2e-87c7-8d38d517bd2e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.891719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.892855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.893384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.893909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.923773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c79533-ed1b-4df8-bf4d-ec886f1f3d7e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:21.948894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:21.960828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:21.961713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:21.962020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:21.962349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:21.970421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.971956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.984205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:21.985942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:21.986127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:21.986984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:21.987876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 383/752] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:21.988361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.084305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.102153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952109-5b53-4109-8773-22a3c565b2e5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.106008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 370/753] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:22.107070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.157480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.159584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.159872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.160151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.161011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.161330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.163601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.189826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 384/754] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.189826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.231095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630482-6d40-4994-be29-862a2f0e218a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.236349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.275139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.275808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.276103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.276368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.277173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.277481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.279428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.297817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 371/755] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.299037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.309568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721bb33-4d61-4bb9-8971-0680d64ff91d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.346605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.347778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.348316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.348818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.349934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.362432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.364462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:22.385214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:22.386232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:22.387206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:22.387206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:22.387206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:22.387206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:22.393870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 385/756] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:22.395031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.413104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b65548-2316-4f9e-b3be-22d7ed94333c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.463722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.464626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.464934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.465265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.480853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.482520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:22.502734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:22.503696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:22.504654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:22.504654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:22.504654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:22.504654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:22.508828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 372/757] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:22.510034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.516102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.525340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8219c-babd-4139-9eb1-229d1b1159ee None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.564366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.565531 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.566164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.566645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.567690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cd1fea13991f4f4b8369cfad704259b9) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.568206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.570242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.576859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 386/758] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/role_assignments?user.id=cd1fea13991f4f4b8369cfad704259b9&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.578167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.683314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:22.695987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a28e5-91c7-43bb-9e35-ee404bc33207 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.705487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be4ad-963f-4d9c-abde-a89c8fc0483a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:22.774214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.774214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.774214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.774214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.774594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.774594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.774594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.798629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 387/759] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.804780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.807726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:22.810261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:22.810261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:22.810261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:22.810261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:22.810261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:22.811411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:22.843777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 373/760] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:22.848206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:22.853996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.036110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3c02-827e-464f-a3b8-71f2c3ee8bcb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.073516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63038b-a6aa-4639-9978-534c11a43a98 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.116239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.117215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.117509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.117821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.126136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.126949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.127613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.128218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.128703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.129292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:23.143152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:23.144095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:23.144985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 374/761] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:23.150230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.152021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.163182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.172137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:23.209239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:23.210262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:23.211165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:23.213774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 388/762] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:23.220624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.262161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398520e-64bb-4fd9-b4df-66d0646b1e31 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.304294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.305043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.305435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.313942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.314903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.315235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.316688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.343002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 375/763] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:23.349427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.377687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.413689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9053-b534-4e9b-962d-4b07a4c22298 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.480027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.496249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.496959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.497252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.497595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.498654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.499188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.502090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.531391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bdb78-0305-4dc7-bb0d-7302f77f0271 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.551863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 389/764] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:23.558342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.602137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.602820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.603306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.603719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.605149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cdf6e3c71de545d782c99ca87d86fb1e) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:23.605573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.607436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc25888-eb93-48ef-b545-24a038c83b0e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.616855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 376/765] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/role_assignments?user.id=cdf6e3c71de545d782c99ca87d86fb1e&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:23.622442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157516-7d88-477a-a7f2-ea8f531bad2e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.623180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157516-7d88-477a-a7f2-ea8f531bad2e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.623530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157516-7d88-477a-a7f2-ea8f531bad2e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.625257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 377/766] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:23.631311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b30c-c94e-4c76-90dd-d0efbd8cc7cc None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.631502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b30c-c94e-4c76-90dd-d0efbd8cc7cc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.631763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b30c-c94e-4c76-90dd-d0efbd8cc7cc None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.637107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 378/767] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:23.637107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b9665-90be-49c1-b217-93719ccb0318 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.637107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b9665-90be-49c1-b217-93719ccb0318 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.637107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b9665-90be-49c1-b217-93719ccb0318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.655017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1c9b9665-90be-49c1-b217-93719ccb0318 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:23.656174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1c9b9665-90be-49c1-b217-93719ccb0318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:23.697690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb243d9-54c1-419a-af6a-27f6b1342b67 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:23.740213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:23.741141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.741428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.741736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/f5953aa40eb54d399b593000f3c74596/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.743781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=f5953aa40eb54d399b593000f3c74596, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:23.745292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:23.749821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 379/768] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:23.752648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:23.753257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:23.753557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:23.756225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.771626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 390/769] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:23 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/f5953aa40eb54d399b593000f3c74596/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:23.785311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.805038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:23.807494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:23.906389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:23.909864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 380/770] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:23.932552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:23.975521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a3c62-9e4c-408a-96eb-75e3fb7be38d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.030716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.032311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.033306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.033833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.034989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.037651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:24.054574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:24.055597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:24.056441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:24.059603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 391/771] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:24.068472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.101821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc154f86-8f51-444d-851b-e0cb3b6f6985 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.148889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.206469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222710e-a694-40bc-a87d-15980096f797 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.225468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.226558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.226851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.227134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] PATH_INFO: `/v3/services/network` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.227921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.229331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.245801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.247035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.247486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.247906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.248806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:24.249259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.251114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:24.255706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:24.256889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:24.257884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:24.263643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 392/772] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:24.265016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.266629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 381/773] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:23 2022] GET /identity/v3/services/network => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:24.272578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.388566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.483219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1e548-3a6b-4e9b-a644-50a8b93f2bad None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.496537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916504d2-9e09-4b7f-91d2-aeac2b6f4441 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.554021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.561506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.562219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.562508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.562860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.565624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] RBAC: Adding query filter params (name=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:24.565952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.567346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.571678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.572370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.572660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.572945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.573811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:24.574132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.575517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.584064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 382/774] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:24.586111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.590375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 393/775] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:24.595475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.759788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b41ec-40c9-4a19-adab-d758f4a99795 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.790607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:24.809013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.809710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.810004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.810287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.811014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] RBAC: Adding query filter params (type=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:24.811331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.812935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.825098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 383/776] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:24.831190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:24.851048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b8039-ac80-42a5-b0a1-4326b538f840 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:24.933979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:24.934685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:24.934976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:24.935260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:24.936129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] RBAC: Adding query filter params (interface=public, service_id=c17de086147f4baea6336bc0e149bece, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:24.936459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:24.937985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:24.957071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 394/777] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/endpoints?service_id=c17de086147f4baea6336bc0e149bece&interface=public®ion_id=RegionOne => generated 194 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:24.962436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.004928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53b425-9928-4c6f-b027-21876dc50018 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.067781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.068477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.068770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.069052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.069905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] RBAC: Adding query filter params (interface=public, service_id=27ac826396ee4ffea7b323f9c2ca4981, region_id=RegionOne) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:25.070012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.070230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.073946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.096578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 384/778] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/endpoints?service_id=27ac826396ee4ffea7b323f9c2ca4981&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:25.101691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.176151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e41fc7-d4f4-41b8-a8a3-f3de7f530527 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.177063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.218676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740a9c0-fcb8-4225-8190-d1e8c5dd0e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.220238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740a9c0-fcb8-4225-8190-d1e8c5dd0e1a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.220733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740a9c0-fcb8-4225-8190-d1e8c5dd0e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.221220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740a9c0-fcb8-4225-8190-d1e8c5dd0e1a None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.222168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740a9c0-fcb8-4225-8190-d1e8c5dd0e1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.223782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740a9c0-fcb8-4225-8190-d1e8c5dd0e1a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.239395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32d6e4-27e1-45c9-9b21-d6881a529a1a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.240056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 395/779] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:24 2022] GET /identity/v3/services => generated 772 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:25.241857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed8974-ea03-4e2c-a5f5-a7729885c7a9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.242354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed8974-ea03-4e2c-a5f5-a7729885c7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.242881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed8974-ea03-4e2c-a5f5-a7729885c7a9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.253976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 396/780] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:25.260060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.260656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.261139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.291158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:25.294796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:25.314601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.316207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.316748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.317251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.318266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.321698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.332044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 385/781] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:25.474014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 397/782] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:25.498631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.641960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f329b-7bbe-4d09-84b5-82bcdbc0c1db None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.693176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.727334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.728164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.728448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.728724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.746565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:25.754648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:25.755649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:25.756519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:25.760514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 386/783] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:25.761838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:25.861892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:25.917121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723ee75-401d-4cea-86d2-9e155423a152 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:25.961222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:25.961956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:25.962252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:25.962585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:25.963301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:25.963616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:25.965009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d87fe6-40fb-4846-84cd-aaf7c49d1a6f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:25.974801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 398/784] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:25.980308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.178166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef223db4-ea8f-4171-acd9-6030aa35cace None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.178343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef223db4-ea8f-4171-acd9-6030aa35cace None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.178632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef223db4-ea8f-4171-acd9-6030aa35cace None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.180298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 399/785] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:26.185351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505125a0-d831-41e8-b6d1-db1b75b1a39c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.185648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505125a0-d831-41e8-b6d1-db1b75b1a39c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.185959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505125a0-d831-41e8-b6d1-db1b75b1a39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.191232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47fc7-02f6-45a9-9f9e-097439bdc795 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.230747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-505125a0-d831-41e8-b6d1-db1b75b1a39c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:26.233085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-505125a0-d831-41e8-b6d1-db1b75b1a39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:26.239164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.240272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.240785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.241257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.262785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.264609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:26.301340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:26.302359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:26.303287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:26.303287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:26.303287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:26.303287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:26.311693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 387/786] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:26.318463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.363376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.371484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 400/787] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:26.372948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065ac16-60de-4cee-a1c2-5a3f20fc62c2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.373240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065ac16-60de-4cee-a1c2-5a3f20fc62c2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.373520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065ac16-60de-4cee-a1c2-5a3f20fc62c2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.378997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 401/788] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:26.386986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.387283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.387572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.404438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:26.405678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:26.443392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.455520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175c7aa-03c6-48d3-9315-68c705e2ea5b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.497849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 402/789] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:26.498715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.499714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.501160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.502050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.504043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.504922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:26.505573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.507778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.529227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 388/790] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:26.530640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.623460 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc7db6-e30e-44ac-88f1-b1000fab14b8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.638820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.651685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e81b41-61f0-405e-b80b-4e5ec9d48eaf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.663975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.665135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.665569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.666019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.688795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:26.692574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.693544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.693887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.694268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:26.698881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:26.699815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:26.700654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:26.704157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 403/791] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:26.705172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.709900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:26.718488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:26.720196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:26.721262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:26.728247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 389/792] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:26.729287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.747793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:26.828024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb5d6b-aea3-48ce-a3ec-2a240918f5cc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.850847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a351-4d07-4283-8ba4-8e96cb74c6a0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:26.865041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.866212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.866685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.867142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.875234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.877590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.890440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:26.891522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:26.891949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:26.893081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:26.893865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:26.894181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:26.895391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:26.896306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:26.897292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:26.897866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:26.900385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 404/793] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:26.901740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.910108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 390/794] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:26.910952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:26.929218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.008471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.023778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff99e3a-c696-4abc-b3c3-e988ed5d3543 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.033360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba91ac9-bab3-4ad8-adfd-4af108b2e3f7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.069431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.070507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.071035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.071142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.071627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.071799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.072040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.072318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.073076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:27.073384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.073636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:27.074193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.074793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.076035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.089753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 405/795] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:27.091115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.100913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 391/796] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:27.102197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.182108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.215689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf065a-3274-4543-b182-5785a00688d3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.226038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deb5fd-0964-4d7a-9f70-5e0e5c9aca94 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.263733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.264881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.265368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.265877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.271478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.272285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.272569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.272846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.286793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.286793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.287047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.288448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:27.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:27.305525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:27.306633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:27.306633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:27.306633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:27.306633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:27.306633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 392/797] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:27.307935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:27.308899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:27.309818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:27.310867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:27.310867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:27.310867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:27.310867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:27.317742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 406/798] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:27.318713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.323862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.431458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dfa01-f410-4841-85ad-cc343f898af2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.465425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5544a-c0f7-45a6-98fc-690667285178 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.475185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.489892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.490695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.490977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.491253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.508054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.509502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.511526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.512513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.512993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.513479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.514589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:27.515095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.516742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.530139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 407/799] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:27.530607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:27.531479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:27.533203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:27.533203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:27.533203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:27.533203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:27.534690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.541394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 393/800] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:27.541394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.627333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.659850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216de0b7-61cf-4c98-a09b-c58a4e155eda None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.684047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d10f0-9710-490e-98aa-92957ed52b8c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.721509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.722871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.723718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.724478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.731843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.732673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.732968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.733249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.734126 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:27.734413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.734520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.736280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.736521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.749459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 408/801] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:27.750977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-dc6ea07e-203b-4ad9-8b5e-e341ef5cca7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:27.752741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:27.753725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:30:27.754689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:27.762427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 394/802] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:27.762427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434e86-c652-4305-afd3-2add8f95cee4 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.762427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434e86-c652-4305-afd3-2add8f95cee4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.762427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434e86-c652-4305-afd3-2add8f95cee4 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.762427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 395/803] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:27.762427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434e86-c652-4305-afd3-2add8f95cee4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.805217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.868607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4847b0-3915-4301-8abc-dcde0f461a39 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.873272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434e86-c652-4305-afd3-2add8f95cee4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:27.894837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:27.906367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.907344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.907633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.907913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.908724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:27.908927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:27.909050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.909751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.910042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.910324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.910492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.918018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:27.919438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:27.927173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 409/804] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:27.927173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-04b34c23-aac5-4bb2-8657-c0e9ec3cc450 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:27.935401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:27.936357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:27.937333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 396/805] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:27.938680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f350-06ce-4c26-86a7-0c5dce24a632 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:27.939093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f350-06ce-4c26-86a7-0c5dce24a632 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:27.939093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f350-06ce-4c26-86a7-0c5dce24a632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:27.955279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eb28f350-06ce-4c26-86a7-0c5dce24a632 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:27.956461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eb28f350-06ce-4c26-86a7-0c5dce24a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:28.029713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 397/806] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:28.029713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52c3d-efee-42b9-99e7-950fd0e7ce1d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.029980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7641b-8cb2-4e66-90b1-7985ab382e9e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.030060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52c3d-efee-42b9-99e7-950fd0e7ce1d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.030060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52c3d-efee-42b9-99e7-950fd0e7ce1d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.030224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 398/807] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:28.030791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52c3d-efee-42b9-99e7-950fd0e7ce1d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.061770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.071095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.071773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.072064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.072346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.073186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:28.073503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.074963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.095084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 410/808] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:30:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:28.095973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.143344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.202038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df39ab-f71b-4726-b2b2-77d484583df5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.238256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.238925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.239211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.239494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.240332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:28.240644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.242089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d712668-6bd7-4d03-917f-fc046fe03c05 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.258343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52c3d-efee-42b9-99e7-950fd0e7ce1d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.260366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 411/809] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:28.261691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.262007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.262291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.278290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:28.279477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:28.297450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.298358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.298648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.298929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.305875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.307317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:28.322108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:28.323168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:28.324192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:28.325801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 399/810] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:28.326875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.348606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 412/811] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:28.349491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.380791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.425417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.441276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ab5c6-802b-4a1b-b5a3-39b8fd159fb2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.454695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5682b-4a50-4c2c-9a14-7f30b69d2baf None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.475914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.476735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.477045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.477332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.478031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.479394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.488689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.489335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.489622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.489931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:28.491624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:28.492725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:28.493793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 400/812] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:28.495232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.561524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.563208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.592050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.595491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3e897-beb9-43ca-be10-c9b1ac6e2ff3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.628456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.629259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.629547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.629858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.640887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.642425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:28.660916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:28.662099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:28.663007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:28.663007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:28.663007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:28.663007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:28.663007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 401/813] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:28.663007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.665766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.675044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 413/814] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:28.676400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.778127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daceaf2b-919a-4954-8b22-46801ebc0dc3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.793357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded0f1-f28a-4249-9192-0a8c8e49265c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.811084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.811756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.812047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.812329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.813176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:28.813494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.814947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.822198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.827113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 402/815] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:28.828359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.830781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.831597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.831910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.832193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.833187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.835485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:28.849505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:28.850553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:28.851404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 414/816] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:28.852220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:28.898651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:28.945708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03885fc-dacd-48db-8a62-479cc15fa4d0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.977864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d6f1-fa94-4e12-a3a3-eedc43a37905 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:28.992730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:28.993406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:28.993723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:28.994009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:28.994727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:28.995034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:28.996386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.001624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 403/817] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.002544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.026871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.027573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.027878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.028122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.028866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.029177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.030773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.055354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 415/818] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.056695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.124361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.129103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411b257-1b7d-44af-86af-fcb67d117414 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.158380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.170346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.171258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.171548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.171830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.181408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550903d7-e542-4fa8-9911-c14732207c9e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.183432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.185050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:29.206426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:29.207372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:29.208266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:29.208266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:29.208266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:29.208266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:29.208266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 404/819] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:29.208547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.221643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.222687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.222975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.223258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cd1fea13991f4f4b8369cfad704259b9/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.242255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=cd1fea13991f4f4b8369cfad704259b9, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.243740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.258547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 416/820] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:29 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cd1fea13991f4f4b8369cfad704259b9/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:29.259539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.350147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723ad15-80a7-4d43-8bff-1b29d81a1074 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.397891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.398944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.399242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.399522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.400261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] RBAC: Adding query filter params (name=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.400580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.402363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.406981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a38d62-b125-445b-a2b1-5b451853456f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.410022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 405/821] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.410697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.411529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.474282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.475063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.475824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.475824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.476597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.476983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.478605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.480429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.487186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 417/822] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.488312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.565580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0858-059e-49f3-982f-d1e09e005c30 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.616855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.618049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.618527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.618814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.625183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be79e-298f-4539-8c86-15fb3e591ef1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.627132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.629752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:29.647781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:29.648774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 406/823] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:29.654775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.671124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.673304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.673875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.674872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.676707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.677438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.680828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.697006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 418/824] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.698231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.731767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.800823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e0cd1-a5b1-408a-8295-64f89eec0306 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:29.802991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:29.849884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:29.850970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:29.851722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:29.852224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:29.853281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] RBAC: Adding query filter params (type=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:29.853905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:29.855423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:29.868358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 407/825] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:29.869553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:29.964841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80599c1-ebee-413d-9407-75bc6a66ffb0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.006578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db99cf9-8c4b-41a7-a130-8e91f05f1d2c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.007078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.008103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.008278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.008673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.009503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.010982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.041652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 419/826] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:30.042623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.086688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.087404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.087683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.087953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.088768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.089085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.090612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.123100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 408/827] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:30.124436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.164854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.218633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.250109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd146f-b559-4186-b314-a5d2192c59d8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.273614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa5f22-9f15-40ae-918e-8edc1c45ee3b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.305374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.305374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.305646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.305825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.316543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.318134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.318451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.319231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.319527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.319813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.326726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.328405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:30.333885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:30.334795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:30.335885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 420/828] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:30.341390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.376701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 409/829] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:30.383489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.494469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97a7c1-6927-4ff9-9733-4fd870f12ffe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.551938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.552862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.553148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.553429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cdf6e3c71de545d782c99ca87d86fb1e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.555761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=cdf6e3c71de545d782c99ca87d86fb1e, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.558212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.580680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.584996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:30.593460 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 421/830] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:30 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cdf6e3c71de545d782c99ca87d86fb1e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:30.632678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d47a7c-ab5f-4e41-87b6-11b4807475b6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.677702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.678405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.678695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.678974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.679884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.680203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.682853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bfe8e-b470-45f5-bce3-043c680da29f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.700683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 410/831] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:30.707902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:30.823147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd274f1-ddfd-40bc-a017-dddf24fedec3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:30.870986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:30.871669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:30.871954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:30.872233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:30.873085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=f5953aa40eb54d399b593000f3c74596) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:30.873404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:30.876581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a29b28-eaf8-4f55-94a3-83810e55c63d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:30.883838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 422/832] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:30 2022] GET /identity/v3/role_assignments?user.id=f5953aa40eb54d399b593000f3c74596&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:31.709107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17824039-5685-469f-84f1-046c6f68eac5 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.709625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17824039-5685-469f-84f1-046c6f68eac5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.709625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17824039-5685-469f-84f1-046c6f68eac5 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.711262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 411/833] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:31.720818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:31.721301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:31.721411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:31.744887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:31.746260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:31.811913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:31.824330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 423/834] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:31.854056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17824039-5685-469f-84f1-046c6f68eac5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:31.984501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17824039-5685-469f-84f1-046c6f68eac5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.024110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.025276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.025846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.026333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.034798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.041499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:32.050889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.051911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:30:32.052757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:32.055588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 412/835] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:30:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.063013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.213589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4838787-2fe0-4225-831a-697148acdee3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.260762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.261367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.261653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.262991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.265601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:32.268310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.269801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a937b-f200-4f7f-8d55-4533fd82036e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.284963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 424/836] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.290475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.432223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15370e6-2c66-40be-a972-0cd6abf62131 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.485693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.485693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.485693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.485693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.497706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.497706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:32.517334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.518298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.519206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:32.519206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:32.519206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:32.519206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:32.519206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 413/837] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.519992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b37f7-9937-4584-b24d-f3444a9d8005 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.519992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b37f7-9937-4584-b24d-f3444a9d8005 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.520292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b37f7-9937-4584-b24d-f3444a9d8005 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.526290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 425/838] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:32.528016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.531094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.531392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.531875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.563999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:32.565200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:32.588177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.647355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d290d-e25a-4e13-b9d1-76083ba9b315 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.672929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.691760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.692465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.692727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.693004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.693763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:32.695472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.696864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.706743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 426/839] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:32.717697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 414/840] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:32.719706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.738763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.831311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5829d47-ba7a-45fa-a4c8-e16c4e747e90 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.873649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a90cae-011c-41b3-aced-143ae79272e9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:32.880844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.884095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.884584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.885070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.889451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:32.898855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.901034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:32.921469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.923189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:32.924376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:32.931945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 427/841] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.933407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:32.938688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:32.939519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:32.939915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:32.940270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:32.941014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:32.944532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:32.961848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:32.962908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:32.963764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:32.968388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 415/842] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:32.970101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.000799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.075268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e9de4-c44d-4bcc-8224-4879c1595ac4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.154475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.155300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.155902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.156155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.160380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:33.160840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.162429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4dab7-3ea2-486b-8527-abbcaabb4879 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.163243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1acc1b-1fa6-45ec-8928-5bd673f61c70 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.192607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 428/843] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.194470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fe0a2-922a-4a22-a892-4d055bba3c2e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.194852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fe0a2-922a-4a22-a892-4d055bba3c2e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.195233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fe0a2-922a-4a22-a892-4d055bba3c2e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.197439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 429/844] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:33.198466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fe0a2-922a-4a22-a892-4d055bba3c2e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.209747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.210806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.214424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.214796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.215617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:33.216066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.217789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb87cd7-d379-4c60-9ec1-2d584f294e86 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.225342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 416/845] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.226729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.227031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.227323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.252473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:33.253360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:33.294524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.341708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 417/846] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:33.342084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.367715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fe0a2-922a-4a22-a892-4d055bba3c2e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.371290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.415028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf41259-1e46-4979-933d-edf2f65404f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.415617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf41259-1e46-4979-933d-edf2f65404f0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.415902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf41259-1e46-4979-933d-edf2f65404f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.416185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf41259-1e46-4979-933d-edf2f65404f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.471924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d263bd-47ac-4fa6-b242-45705eec066c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.497080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf41259-1e46-4979-933d-edf2f65404f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.498564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf41259-1e46-4979-933d-edf2f65404f0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.512401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.513400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.513944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.514472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.515206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:33.515526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.516941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.522439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 418/847] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:33.523248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.564757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.630493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 430/848] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:33.632287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47e02d-4a7e-47c6-b693-f0c76dc758e2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.632587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47e02d-4a7e-47c6-b693-f0c76dc758e2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.632874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47e02d-4a7e-47c6-b693-f0c76dc758e2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.637886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 431/849] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:33.638845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47e02d-4a7e-47c6-b693-f0c76dc758e2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.644126 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.668111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e15c0-3768-43e1-bc4b-440766b622d3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.720005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c456f-99b9-406f-b424-70e73c2d4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.721678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c456f-99b9-406f-b424-70e73c2d4383 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.722149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c456f-99b9-406f-b424-70e73c2d4383 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.722600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c456f-99b9-406f-b424-70e73c2d4383 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.733996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c456f-99b9-406f-b424-70e73c2d4383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b32c456f-99b9-406f-b424-70e73c2d4383 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:33.742904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:33.743877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:30:33.744707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:33.747273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 419/850] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:33.748956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.749469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.750021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.770514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:33.772271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:33.774522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47e02d-4a7e-47c6-b693-f0c76dc758e2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:33.814060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:33.815060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:33.815351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:33.815629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:33.835006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:33.838105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:33.863148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:33.864105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:33.865479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:33.865479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:33.865479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:33.865479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:33.872148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 432/851] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:33.873290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.873990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 420/852] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:33.875060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:33.920656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.949648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:33.998614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e985d77-12e7-4619-a89b-8148eba08d47 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.005565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241034ed-2bf2-448a-8613-2895155c9567 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.040107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.040793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.041106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.041374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.042174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.042490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.043873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.055187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.056311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.057090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.057583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.057735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 433/853] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.058634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.058912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.059458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.061324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.078076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 421/854] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.079375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.172127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.172731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623a5d2-7e1a-441a-9399-1de2c39d9580 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.192479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.192937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dacf7-77ef-43df-8ecd-baab1d6a9acf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.221188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.222053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.222336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.222610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.223268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.224627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.237486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.238327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.238616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.238896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:34.239267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:34.240186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:34.241040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:34.241919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 434/855] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:34.242640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.260207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.262067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:34.288755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:34.289759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:34.290720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:34.290720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:34.290720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:34.290720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:34.293843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 422/856] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:34.294696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.382705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3aac7f-10b0-4fe0-bdbb-33bd0b7ca1e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.424347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.425683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.427176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.427176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.439020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.441600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:34.469626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:34.470658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:34.471558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:34.479085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 435/857] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:34.480528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.483466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694754-0c0d-4e42-a087-b92a042c5273 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.525243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.526333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.526986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.527485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.528769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.529290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.530974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.533144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.538811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 423/858] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.539896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.550241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.623322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a62228-96b2-460f-b998-bb89129f6a9b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.662411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80459e-b1ae-45ad-8456-f77bd287baf5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.663638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.664327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.664626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.664951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.665938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.666257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.667797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.681898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 436/859] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.682973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.709772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.710415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.710721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.711277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.712116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.712673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.714092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.729559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 424/860] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.730634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.815801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.826679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f5999-1e0f-43ed-9333-5557957ba79b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.832445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:34.869682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.870319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7c07-8396-49d9-9f2e-c21561633f88 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:34.871265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.871868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.872346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.873369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:34.873916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.879224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.889840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 437/861] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:34.890806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:34.924536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:34.925363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:34.925786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:34.925981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:34.934642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:34.936059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:34.952887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:34.954506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:34.955588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 425/862] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:34.965432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.042597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dafe30-637a-40dd-8453-9beb5033adb5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.113972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.114768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.115404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.115404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.116652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cd1fea13991f4f4b8369cfad704259b9) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:35.116652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.118182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.135214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 438/863] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/role_assignments?user.id=cd1fea13991f4f4b8369cfad704259b9&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:35.137179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c689b-9bfb-4e84-8761-dcc107e3b396 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.161789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.182180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.183318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.183809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.184290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.185419 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:35.185990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.187695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.212017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 426/864] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Jun 15 14:30:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:35.221712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.278144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.354401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d2239-01ff-4f4a-947d-caf649f29559 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.400593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d4b67-9fd7-4660-97d3-85ea32f670c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.401612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d4b67-9fd7-4660-97d3-85ea32f670c8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.402134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d4b67-9fd7-4660-97d3-85ea32f670c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.402612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d4b67-9fd7-4660-97d3-85ea32f670c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.488182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d4b67-9fd7-4660-97d3-85ea32f670c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.490048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d4b67-9fd7-4660-97d3-85ea32f670c8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.522852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.608051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 439/865] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:35.614789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.758828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94356cf-64f7-496b-8851-680ffd0635f1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:35.759436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab0017-9859-402c-8e9a-3400fd82dd7c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.759532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab0017-9859-402c-8e9a-3400fd82dd7c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.759802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab0017-9859-402c-8e9a-3400fd82dd7c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.761456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 440/866] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:35.767824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.768116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.768403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.795226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:35.796469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:35.801394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:35.802269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:35.805683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:35.805683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:35.825580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:35.828838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:35.859719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:35.860865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:35.861888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:35.861888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:35.861888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:35.861888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:35.861888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 427/867] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:35.865686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:35.924569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:35.934803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 441/868] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:35.959165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.045682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a8a9-b8b6-4660-afa8-b936b16aeee0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.110396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.120872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.121560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.122014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.125793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.125793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:36.125793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.125793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.131272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7697ad5-cbe7-4c81-8302-e819a0579cc8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.140206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 428/869] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:36.151444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.206470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.209694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.209694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.209694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.209694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.210186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.246593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 442/870] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:35 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:36.269466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.319799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d7c-9328-4265-b797-83f5cfaca670 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.376181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.377433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.377433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.377433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.386397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.387878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:36.404169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:36.405136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:36.406056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 429/871] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:36.413348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.535133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1933b-1bca-4d2c-9bd1-5783edb5e4cb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.561183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:36.582261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.583011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.583331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.583697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.584571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:36.584958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.586484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ef7e3-72eb-4ebb-9e82-20b0fc860386 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.600379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 443/872] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:36.606224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:36.740371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fa26c-1df0-47e1-a17e-4f9e0451cda9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:36.777421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:36.778120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:36.778392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:36.782029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:36.782029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cdf6e3c71de545d782c99ca87d86fb1e) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:36.782029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:36.782029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d689ca-ba2b-49ea-a5b0-de21a963814e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:36.788491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 430/873] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:36 2022] GET /identity/v3/role_assignments?user.id=cdf6e3c71de545d782c99ca87d86fb1e&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:37.416746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895e29a-eb37-47f1-abb8-a5210503cf37 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.417352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895e29a-eb37-47f1-abb8-a5210503cf37 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.417352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895e29a-eb37-47f1-abb8-a5210503cf37 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.418891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 444/874] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:37.427285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89bc9d-a4ee-4f4a-b195-aa55248b4c5c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.427285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89bc9d-a4ee-4f4a-b195-aa55248b4c5c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.427285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89bc9d-a4ee-4f4a-b195-aa55248b4c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.460454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3e89bc9d-a4ee-4f4a-b195-aa55248b4c5c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:37.461946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3e89bc9d-a4ee-4f4a-b195-aa55248b4c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:37.465756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:37.544284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 431/875] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:37.579361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895e29a-eb37-47f1-abb8-a5210503cf37 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:37.718682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc6086-2c49-46cf-8d3f-712634dab68f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.719295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc6086-2c49-46cf-8d3f-712634dab68f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.719989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc6086-2c49-46cf-8d3f-712634dab68f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.727658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 432/876] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:37.731148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895e29a-eb37-47f1-abb8-a5210503cf37 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:37.733251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.733527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.733928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.753489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:37.756092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:37.777496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:37.779773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:37.779773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:37.779773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:37.781327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:37.783471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:37.813688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 445/877] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/services => generated 269 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:37.877554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 433/878] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:37.901136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:37.950968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.017723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fe9aa-6489-4af3-b647-3f1b87ad4dbb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.053594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.054453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.054741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.055025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.067675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:38.076536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.078119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:38.079311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 446/879] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:38.087379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.159710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.207310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3700788-899f-42f3-818f-c18e94a6633b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.244584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.245277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.245568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.245886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.246646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:38.246963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.248400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.258774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 434/880] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.266179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.377010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549c5e9-4be9-43f5-ba93-94f56d4d83e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.380165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.414139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.415054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.415352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.415636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.428802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.430311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:38.458080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.459036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.459986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:38.459986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:38.459986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:38.459986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:38.459986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 447/881] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:38.465474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.576377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fef03-a218-4bdd-ab88-2c26a145f0fc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.606146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.614281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.614987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.615279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.615585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.616406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:38.616745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.618254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537a115-5c76-488e-94b0-d65d8b3858b2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.633322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 435/882] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:38.638965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.756161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8959a6-2860-4c6c-8b78-9bd6ce7eaa59 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:38.783592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab9e18-4eb6-41c3-a54a-8dcc67d2437b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.784234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab9e18-4eb6-41c3-a54a-8dcc67d2437b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.784721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab9e18-4eb6-41c3-a54a-8dcc67d2437b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.790441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 436/883] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:38.792300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.792811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.793405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.793798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:38.794629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:38.794918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:38.795199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:38.811748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:38.811748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:38.818337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:38.824617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:38.838014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:38.839026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:38.839984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 448/884] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:38.847683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:38.878335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:38.941979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 437/885] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:38.981695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.037688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fffab2-30f4-43ec-967c-856ef30a3b8f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.097709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.097709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.097709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.097709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.102147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.102147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.105689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.144467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 449/886] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.150912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.154310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375700c2-ff3f-4aea-85e7-ed100c37bcc1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.180675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.212264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.213830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.214179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.214640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.215361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.216787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4c9cad9e-2efe-48a6-a9ad-5dfa11440031 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.232855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.234002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:39.240589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 438/887] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:39.241408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f3496-d19a-4e94-b395-be8fe0af787a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.241408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f3496-d19a-4e94-b395-be8fe0af787a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.241408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f3496-d19a-4e94-b395-be8fe0af787a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.242430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 439/888] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:39.244340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f3496-d19a-4e94-b395-be8fe0af787a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.289331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.324110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30ea1c-daa7-4f33-b084-528cb2554c54 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.378543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.379570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.380028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.380664 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.382900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f3496-d19a-4e94-b395-be8fe0af787a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.420607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.421572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.422783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.423391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.427214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.427887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.429507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80e35e-2a7e-4e8e-87e9-592b775d184f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.441361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 440/889] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.441361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.441361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.441361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.472770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:39.474369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:39.478360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.480090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.523806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.560614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 441/890] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:39.562608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.581281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.631461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 450/891] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:39.632877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.693712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a62505-1909-42c8-8226-6b31079bb420 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.745530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.746361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.746652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.746937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.747660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:39.747974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.749342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.756441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38df6b-656b-42cb-a1e4-af37fd08763f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.760668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 442/892] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:39.761826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.798723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.799947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.800450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.800959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.801953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.803646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.810021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.819220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.820501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:39.821565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:39.829189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 451/893] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:39.829189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:39.868853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:39.889232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daf9ff-50d2-4d9f-98ee-116d9c262b45 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.935308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:39.936741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:39.937037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:39.937324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:39.950770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:39.953087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:39.964133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf392706-a787-4c63-81e5-8d4b57631824 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:39.973972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:39.974904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:39.975807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:39.975807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:39.975807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:39.975807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:39.976026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 443/894] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:39.976790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.010448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.011417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.011919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.012404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.013215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] RBAC: Adding query filter params (interface=public, service_id=21c515d18d194fdcb6210138fbf5fdb0, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.013538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.015018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.021739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 452/895] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/endpoints?service_id=21c515d18d194fdcb6210138fbf5fdb0&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.022538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.090548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.105078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8893648-a48d-4ca1-9880-c6f5b61c13ca None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.142260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.147663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a03847-7785-4040-9cdd-2c8c13bcb2f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.153809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.154495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.154785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.155067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.155798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.156112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.158247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.166536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 444/896] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.167740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.202057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.203901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.204542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.205034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.205898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.206292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.207893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.223454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 453/897] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.224366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.302399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc45c-1217-4faa-ac50-18792cb9bc30 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.339826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.340506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.340799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.341088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.341841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.343209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.350473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 445/898] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/services => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:40.351366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.353030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7a4d-2598-47fc-9834-10d261b54d50 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.373422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.404881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.405990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.406463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.407726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.408703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.409099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.410805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67caf72-dd94-4ddb-ae12-9e27c88b195e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.425733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 454/899] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.425733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce31ad-4ee6-433e-91b3-2d921dc21497 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.425733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce31ad-4ee6-433e-91b3-2d921dc21497 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.425733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce31ad-4ee6-433e-91b3-2d921dc21497 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.425733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 455/900] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:40.426280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.426550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.426683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.457705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:40.458746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:40.492923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.535829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58d861-914a-4ac8-b885-64428800b889 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.581719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 456/901] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:40.603957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.605576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.606148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.606278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.606808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.617331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.619073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:40.641685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:40.642785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:40.643686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 446/902] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:40.646202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.758040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:40.788082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fb79-fc62-45a5-981b-59ab1c08294a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.817688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539882-053c-482f-845a-81bdd4c9ade6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:40.836566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.837228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.841683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.841683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.841683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:40.841683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.843382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.866820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 447/903] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:40.874220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.893592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:40.895189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:40.895482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:40.895760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:40.896728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:40.898740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:40.915675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:40.916704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:40.917566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:40.925684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 457/904] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:40.925684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:40.961793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.116718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7656ddc-3bd9-44fa-a392-1f189d8437c8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.157688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cb4a3-01a2-4996-bb64-e39e104ab774 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.216320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.221681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.221681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.221681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.221681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] RBAC: Adding query filter params (role.id=31445885c860449bb44473c2cae11fe3, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cd1fea13991f4f4b8369cfad704259b9) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:41.222345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.226155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.245686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 448/905] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/role_assignments?user.id=cd1fea13991f4f4b8369cfad704259b9&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=31445885c860449bb44473c2cae11fe3 => generated 259 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.269686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.269686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.269686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.270360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.273685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:41.273685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.273685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.292549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 458/906] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.297902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.302407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.465725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751965e4-0105-4cb2-8651-288d8a1e0003 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.508136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.510579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.510579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.510579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.510579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:41.510579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.511904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.521783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 449/907] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.523362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.611932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.634058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d97fa-dbba-4374-bfc3-e180ab4a69e1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.670359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.671057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.671413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.671699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.673157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] RBAC: Adding query filter params (interface=public, service_id=34b71a3bc43c460e9bd05b0095c026f5, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:41.673482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.681768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469256f-cd20-4948-8fa7-3ef76676da19 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.685806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 459/908] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/endpoints?service_id=34b71a3bc43c460e9bd05b0095c026f5&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:41.689175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:41.786258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:41.821119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c8d4c-a6d0-4ac2-ad83-b47f6d9ec8e7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:41.856812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49c719-2986-495a-9c80-2d1e128a8470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:41.857496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c719-2986-495a-9c80-2d1e128a8470 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:41.857838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c719-2986-495a-9c80-2d1e128a8470 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:41.858098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c719-2986-495a-9c80-2d1e128a8470 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:41.858793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c719-2986-495a-9c80-2d1e128a8470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:41.860308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c719-2986-495a-9c80-2d1e128a8470 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:41.870340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 450/909] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:41 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:42.858873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80838945-ee45-42fa-955f-5bf292e43932 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.859424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80838945-ee45-42fa-955f-5bf292e43932 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.859424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80838945-ee45-42fa-955f-5bf292e43932 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.861679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 460/910] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:42.867239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.867747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.867828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.887221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:42.888461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:42.949079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e584-8678-49c2-b3d5-2d2470fc5dc9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.949429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e584-8678-49c2-b3d5-2d2470fc5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.949641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e584-8678-49c2-b3d5-2d2470fc5dc9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.951161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 461/911] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:42.957292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:42.957583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:42.958119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:42.974125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 451/912] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:42.979231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:42.980425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:42.997508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.057540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.061801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 462/913] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:43.089083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.119865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bd292-02c9-4866-8e56-d7bc73ed90e3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.156037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.157258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.157786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.158279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.159134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.160698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.170070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:43.174500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:43.175504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:30:43.176339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:43.183706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 452/914] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:43.185242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.199575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccba55-40f1-4070-a919-a544e42ae4d3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.236127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.236803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.237091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.237699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.238172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.245484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.266090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 463/915] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:43 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:43.299697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e716f6-4af5-4f8e-bc0c-5f5e74c3688e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.326485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.345455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.346242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.346533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.346814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.347546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:43.347861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.349258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.359678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 453/916] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:43.362081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.484115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c610283-c366-4dd4-965c-cb01ef34d1d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.521773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.522762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.523231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.523689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.524768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:43.525276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.526905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e99f5-1600-4dbc-a8da-79260d9dd53b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.528944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.534759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 464/917] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Jun 15 14:30:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:43.540033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.659219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44d8e4-5ba2-447f-8481-629d9a2183c5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:43.677333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb828b2-c0d2-44d9-997b-a0368d6537ff None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.677642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb828b2-c0d2-44d9-997b-a0368d6537ff None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.677948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb828b2-c0d2-44d9-997b-a0368d6537ff None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.679574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 465/918] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:43.684430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.684728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.685013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.695652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:43.696312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:43.696597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:43.696876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:43.697836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:43.699202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:43.703185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:43.705814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:43.722938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 454/919] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:43.812295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 466/920] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:43.842413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:43.903427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:43.987653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033d989-ae3c-4f98-ba10-2ad82100bfa7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.029746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.029746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.029746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.030101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.041310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:44.051139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:44.052135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:44.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 455/921] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:44.059022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.194955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1526433-7ef4-4da1-b9c9-6390740584bc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.222125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.241936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.242616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.242911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.245683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.245683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:44.245683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.245683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.255693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 467/922] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.262828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.424566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.429341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034e92-44be-458d-962e-fddecd2bd5ee None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.480216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.481448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.481981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.482458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.496488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.498334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-8dd3738f-5803-4ed7-acd4-d78343d34619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:44.519558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:44.520597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:44.521498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:44.521498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:44.521498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:44.521498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:44.523768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 456/923] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:44.529796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.550656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ce9d-3586-4d84-9829-0981f7f2022e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.551296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ce9d-3586-4d84-9829-0981f7f2022e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.551791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ce9d-3586-4d84-9829-0981f7f2022e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.553534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 457/924] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:44.560720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.561156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.561578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.580545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:44.581930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:44.659895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524cbf-ab5b-4b31-af3a-8a4c727dd62e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.669764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 458/925] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:44.690936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.704142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.704835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.705127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.705413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.706226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:44.706610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.708242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.722679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 468/926] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:44.730809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.783401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.802005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:44.811068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70c8eb-c925-4950-acb8-5f86f9a0d19d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.848564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde8403-848e-4bf1-b4b6-a9157f866775 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:44.855088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.856139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.856623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.857079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.858023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.859752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:44.872379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:44.873310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:30:44.874184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:44.877556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 459/927] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:44.883836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:44.894440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:44.895316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:44.895610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:44.895890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:44.903432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:44.905169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:44.920688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:44.921648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:44.922810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:44.926950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 469/928] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:44.928275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.017737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9cb7b-3b5c-43ab-a70a-331a4eb20d04 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.032260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af8d52-0e98-4950-b7cd-055dc666ef8e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.060988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.066071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.066936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.067378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.067812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.068676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:45.068996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.070409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.073144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.074043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.074477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.074951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.075772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 460/929] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:45.076157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:45.076703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.078644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.082748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.096373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.105384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 470/930] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:45.106539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.193358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a101-fbff-4593-8ccb-049ded8905a9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.218136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6498e-8f9f-42d7-916a-436e4701bf83 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.230369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.231286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.231600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.231949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.232767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:45.233114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.235286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.245627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 461/931] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:45.246960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.258819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.259884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.260475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.260953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.272475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.274386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:45.293615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:45.294581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:45.295490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:45.295490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:45.295490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:45.295490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:45.302994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 471/932] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:45.304205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.324379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.328665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.356397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338a6c-91c0-4fe2-b704-9cb451b539bc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.395537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.396205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.396494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.396776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.397528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.399193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.419594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092c8a4-5872-48a0-b1dc-c6013af54ede None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.421806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 462/933] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:45.490458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.491572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.492046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.492503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.493465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:45.493991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.495609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468b212-2c5b-4562-b799-03024adfc150 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.509871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 472/934] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:45.515839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.634374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0628a-7894-4b22-8edf-d7f4c20b4a4b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.667996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.675012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.676280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60affbaf-7dd1-496e-a859-34da5ad8a08e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.676500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.676624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60affbaf-7dd1-496e-a859-34da5ad8a08e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.676863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60affbaf-7dd1-496e-a859-34da5ad8a08e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.677125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.677411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.678584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 473/935] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:45.683592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.683891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.684173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.687557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.690085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.702501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:45.704547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:45.706844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:45.707908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:45.708992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:45.711054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 463/936] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:45.716735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.783054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 474/937] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:45.801900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.804002 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.825098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be133c1f-1d60-44c7-9bb9-c8179ca2e9e3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.863195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.863852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.864143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.864429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.865229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:45.865550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.867066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.885186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 464/938] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:45.886020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.913511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7ed76-a984-43a9-ac7b-20acfa73c7d6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:45.939531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:45.951949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:45.953326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:45.953949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:45.954586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:45.955403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:45.957042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:45.971494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:45.972772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:45.973724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:45.980716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 475/939] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:45.982179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:45.994792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f526e0-d4d6-4c2b-bb30-8c993935c4f2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.030803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35488dd-9b2b-4a30-b5fd-fede20514743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.031917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35488dd-9b2b-4a30-b5fd-fede20514743 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.032340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35488dd-9b2b-4a30-b5fd-fede20514743 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.032684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35488dd-9b2b-4a30-b5fd-fede20514743 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cd1fea13991f4f4b8369cfad704259b9/roles/31445885c860449bb44473c2cae11fe3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.035136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35488dd-9b2b-4a30-b5fd-fede20514743 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=cd1fea13991f4f4b8369cfad704259b9, role_id=31445885c860449bb44473c2cae11fe3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.036622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35488dd-9b2b-4a30-b5fd-fede20514743 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.053602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 465/940] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:30:45 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/cd1fea13991f4f4b8369cfad704259b9/roles/31445885c860449bb44473c2cae11fe3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:30:46.059692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.102109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c31b-b72a-49fa-9713-706a7d7b7aef None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.110220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f46c86-498d-4288-91da-6f0e964d6f2b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.110561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f46c86-498d-4288-91da-6f0e964d6f2b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.110848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f46c86-498d-4288-91da-6f0e964d6f2b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.113038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 466/941] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:46.118705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.118955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.119238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.138681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:46.139944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:46.143702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.144645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.144953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.145231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.145998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:46.146316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.147787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.158726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 476/942] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:46.160224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.242915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 467/943] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:46.262523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.287746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11fd35-1d8f-465a-ae2b-f7a9c5513bbf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.319807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.334272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.338210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.339540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.339900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.340274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.341355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:46.341767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.343888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.353730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 477/944] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:46.358901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.401087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7f020-c214-48ee-979c-ee2457a8f11e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.459671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.460592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.460956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.461334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.462166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.463767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.465264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING keystone.server.flask.common [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 14:30:46.476179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026528e4-41a8-40f7-9a8e-6c1968a1c4c3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.484628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.sql.core [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:30:46.484628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:30:46.484628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [parameters: {'id': '623bf151962843af9d5b94f19b1f7c11', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:30:46.484628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80379) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 623bf151962843af9d5b94f19b1f7c11.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 623bf151962843af9d5b94f19b1f7c11. Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:46.504789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 14:30:46.505856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application yield resource Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise self.value Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application session.commit() Jun 15 14:30:46.507736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:30:46.509352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:30:46.510539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:46.511547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [parameters: {'id': '623bf151962843af9d5b94f19b1f7c11', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:46.520755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:46.522658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 623bf151962843af9d5b94f19b1f7c11. Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 468/945] 10.222.0.57 () {60 vars in 1140 bytes} [Wed Jun 15 14:30:46 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:46.526024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.526024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.526024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.526024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.526581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] RBAC: Adding query filter params (interface=public, service_id=26dd0fc440024fb0a1b244ec6e46ee63, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:46.526581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.526581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.526985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.527566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 478/946] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/endpoints?service_id=26dd0fc440024fb0a1b244ec6e46ee63&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:46.528606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.592234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.642701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6897c-445a-4f5d-9bb1-f554463b2da3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.661620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9bdc1-61d7-4adf-93d9-a03aafe18f1b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.676809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.681474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.682440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.682964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.683852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.685020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.687468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.701800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 479/947] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:46.706166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.707426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.707917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.708427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.717041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.719245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2512b5f2-b62d-43cf-aea4-12c8f02a1016 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:46.736662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:46.737649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:46.738582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:46.741213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 469/948] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:46.747620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:46.863836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc727e-167e-4124-b2ec-b9f56981dda0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:46.887943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:46.902644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:46.903694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:46.904178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:46.904649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:46.905604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:46.906160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:46.908426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd5143-692c-4343-8dff-167c19b7de92 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:46.926933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 480/949] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:47.862314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d279be-2c7a-449d-9f83-7f2991d0e0bd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.862873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d279be-2c7a-449d-9f83-7f2991d0e0bd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.862873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d279be-2c7a-449d-9f83-7f2991d0e0bd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.867772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 470/950] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:47.874260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:47.874784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:47.874884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:47.893893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:47.895095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:47.989535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 481/951] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:48.012398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d279be-2c7a-449d-9f83-7f2991d0e0bd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.123034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d279be-2c7a-449d-9f83-7f2991d0e0bd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.161332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.162508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.162990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.163512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.164382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.165996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:48.179371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:48.180420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:48.181269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:48.184681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 471/952] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:48.193710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.293819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c7de4-e114-4921-81f9-f5261dcac384 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.307780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.329933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.330615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.330905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.331188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.331905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:48.332219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.333595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3859b9-67ed-43be-b738-1505948b08b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.338792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 482/953] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.344850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.372059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.420726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bbfd7-786c-45a5-a01a-0a3dc5d20b49 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.420921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bbfd7-786c-45a5-a01a-0a3dc5d20b49 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.421211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bbfd7-786c-45a5-a01a-0a3dc5d20b49 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.423544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 483/954] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:48.429231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.429231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.429231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.450230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:48.451876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:48.467905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e893-571b-4017-ace7-2a3ac09368e6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.510721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.511401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.511694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.511976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.512698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:48.513035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.514512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.519953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 472/955] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.533030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 484/956] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:48.556912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.655893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.675632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653a23d-b871-47d1-8feb-af21628662b9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.711843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.712669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.712958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.713239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.725338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:48.733702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:48.735990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:48.736862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 473/957] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:48.741592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:48.847028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ae742-bf15-41cf-8242-6556bea30274 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:48.855127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:48.884975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:48.886008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:48.886328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:48.886615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:48.887387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:48.887728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:48.889158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec4772-24d8-42e5-aa7e-b1ecc983835e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:48.898529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 485/958] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:48.903630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.018634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9c231-8043-4c7e-9230-ce336340546d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.033961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435d195-dbce-421f-bbcc-062247043238 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.034558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435d195-dbce-421f-bbcc-062247043238 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.035075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435d195-dbce-421f-bbcc-062247043238 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.036906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 486/959] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:49.042259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.042796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.043277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.055146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.055951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.056240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.056521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.064193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:49.066103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:49.071558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.073239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.076186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:49.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:49.095306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:49.096213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:49.096213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:49.096213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:49.096213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:49.096448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 474/960] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:49.102390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.151568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 487/961] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:49.174121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.231936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495da4d7-a442-408f-9e25-72192c9c229d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.284204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.284930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.285221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.285505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.287158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:49.287479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.289269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.298122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.303380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b16007-8f75-4f6b-9863-8639c2c95927 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.311540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 475/962] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.317702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.345334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.346732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.347244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.347727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.348626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.350272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cdf46e33-73c4-41a5-bc99-86174d6f9e40 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:49.363790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:49.364881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:49.365805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.367277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 488/963] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:49.368573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd153-eee1-4483-b224-b5a1921ac29d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.368920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd153-eee1-4483-b224-b5a1921ac29d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.369209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd153-eee1-4483-b224-b5a1921ac29d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.373792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 489/964] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:49.375150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfd153-eee1-4483-b224-b5a1921ac29d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.389817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.437433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee9cc-e681-4d70-82d3-b5813ccf210b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.476608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.477457 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.477801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.478085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.486063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfd153-eee1-4483-b224-b5a1921ac29d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.488023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.489464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-376e308e-5ea4-48e4-bb5f-9071de7598be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:49.506274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:49.507230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:49.508189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 476/965] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:49.508820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.509837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.509837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.527104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:49.527666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.528341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.528631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.528741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:49.528922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.529631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:49.529982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.531549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.542292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 490/966] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.542292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.570786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.609703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 477/967] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:49.609703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.662199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0818fc3-ca69-4a31-a6b6-fec1c719feee None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.666490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.701511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.702399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.702585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.703088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.704314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:49.704775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.706104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.730145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 491/968] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.730145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.736599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ca528-e680-4862-ba11-65a94308cf2e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.774835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.775740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.776034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.776319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.777040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:49.777361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.778852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.787111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 478/969] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:49.788162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.823470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.840716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95a1e-baf3-409e-ab1c-84a626daed99 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.874968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.876213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.876507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.876759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:49.877467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:49.878940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:49.880243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING keystone.server.flask.common [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 14:30:49.892850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.sql.core [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:30:49.892850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:30:49.892850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [parameters: {'user_id': 'ff620a14ea914cc599548e0c2fbe7c96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:30:49.892850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80378) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:49.900208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:49.901245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application yield resource Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise self.value Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application session.commit() Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:30:49.902560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:30:49.903488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:30:49.904395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:30:49.905258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ff620a14ea914cc599548e0c2fbe7c96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:49.906164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 14:30:49.908491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 492/970] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/users => generated 160 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:49.910461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:49.911421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224261f-c48c-4d85-9a2d-e74ee48daf22 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:49.920019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:49.947066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:49.947729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:49.948009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:49.948383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.019349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf28fc-fa79-4ac2-9e58-c242b46972e4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.023303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.024999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.055175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.055850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.056141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.056420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.057166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.058759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.089978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 493/971] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:50.090919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.096710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.148079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 479/972] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:30:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:30:50.155142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.171953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.246360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0df526-dafe-47d3-ada6-78428082fbc4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.283703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafdc1f-07f1-4418-864b-4da4fa47d9b1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.290282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.291486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.291803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.292088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.302980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.304444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:30:50.320748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:50.321799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:50.322839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 494/973] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:50.326114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.327602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.328113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.328465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.328912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.341344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.342828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:30:50.368733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:50.370524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:50.371715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:30:50.371715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:30:50.371715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:30:50.371715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:50.371715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 480/974] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:50.377913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.394564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.445135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b89b0-981f-4407-baf4-f71ff3efe3c4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.465199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.490603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.491288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.491583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.491912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.492682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.492997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.494661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.515397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc87b8-0f57-4f67-867d-38cec6d6639c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.535692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 495/975] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:50.556897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.558029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.558556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.559037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.559995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.560532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.562190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b5b97-5198-4b6a-b484-02a888d0da3f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.576672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 481/976] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.582721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.709956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d1361-6dde-44d9-956b-3f2a7ec47c0a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.730438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.732094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd375049-44a5-4549-ab34-7c35383b4e7e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.732481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd375049-44a5-4549-ab34-7c35383b4e7e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.732824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd375049-44a5-4549-ab34-7c35383b4e7e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.734428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 482/977] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:50.741124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.741861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.742356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.750611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.751381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.751672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.751953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.759267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.760909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.767485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:50.768577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:30:50.775898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:50.776794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:50.778006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 496/978] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:50.783503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.861907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 483/979] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:50.884747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.898474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b0188-0b6e-48d3-947e-0d7044baf162 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.937069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:50.937844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:50.938328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:50.938624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:50.939441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:50.939767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:50.941074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:50.941513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:50.954965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 497/980] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:50.960560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:50.991807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2d523-b869-45c1-bf6c-857b2809fedb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:50.996344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:51.032437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9757ce-7ce3-46dd-a30c-8a2b51b5c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:51.033586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9757ce-7ce3-46dd-a30c-8a2b51b5c35b None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:51.033905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9757ce-7ce3-46dd-a30c-8a2b51b5c35b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:51.034187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9757ce-7ce3-46dd-a30c-8a2b51b5c35b None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:51.035063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9757ce-7ce3-46dd-a30c-8a2b51b5c35b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:51.036416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9757ce-7ce3-46dd-a30c-8a2b51b5c35b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:51.051344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 484/981] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:50 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:51.071879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9420f66-bb0c-4c34-a1f8-15ceab5144cb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:51.119380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:51.120060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:51.120346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:51.120631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:51.121581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] RBAC: Adding query filter params (role.id=31445885c860449bb44473c2cae11fe3, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=cd1fea13991f4f4b8369cfad704259b9) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:51.121938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:51.123536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dba5df-2566-4210-ad3d-8171c536b839 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:51.134044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 498/982] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:30:50 2022] GET /identity/v3/role_assignments?user.id=cd1fea13991f4f4b8369cfad704259b9&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=31445885c860449bb44473c2cae11fe3 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:52.364530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db643ec-1c69-4d96-93c4-7780cd54f916 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.365059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db643ec-1c69-4d96-93c4-7780cd54f916 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.365139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db643ec-1c69-4d96-93c4-7780cd54f916 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.366722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 485/983] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:52.372041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.372426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.372617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.391908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:52.393565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:52.472008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 499/984] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:52.495576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db643ec-1c69-4d96-93c4-7780cd54f916 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.602113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db643ec-1c69-4d96-93c4-7780cd54f916 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.615666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.639526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.640344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.640632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.641484 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.642189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.643548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:52.655824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:52.656852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:52.657954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 486/985] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:52.663102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.692965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:52.772219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a315-910b-4b2b-a486-54ad169dc421 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.807539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.808243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.808536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.808820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.809576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:52.809940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.811380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.821857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 500/986] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:52.822885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:52.927263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee05d5-0721-40b3-947e-cc2d7ef6600c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:52.961469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:52.962194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:52.962486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:52.962770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:52.963621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:52.963833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:52.965215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:52.975555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 487/987] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:52.976571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.069565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.080529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21718256-de69-4da8-a25d-81c8fe048df1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.117091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.117833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.118126 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.118439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.119214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] RBAC: Adding query filter params (interface=admin, service_id=26dd0fc440024fb0a1b244ec6e46ee63, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.119534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.120902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2bb0a-aed5-4d6e-abdb-06c83b154d18 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.131191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 501/988] 10.222.0.57 () {56 vars in 1249 bytes} [Wed Jun 15 14:30:52 2022] GET /identity/v3/endpoints?service_id=26dd0fc440024fb0a1b244ec6e46ee63&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:53.132227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.176722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.246087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577087-a640-4b4a-9864-fbbe3acaf153 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.260668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335603-02f8-42f4-a18a-74ae77023b72 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.260973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335603-02f8-42f4-a18a-74ae77023b72 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.261257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335603-02f8-42f4-a18a-74ae77023b72 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.262739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 502/989] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:53.268707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.269016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.269276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.285378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05adb8-1206-45af-897d-3919f3d501bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.286086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05adb8-1206-45af-897d-3919f3d501bf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.286374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05adb8-1206-45af-897d-3919f3d501bf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.286655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05adb8-1206-45af-897d-3919f3d501bf None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.286772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:53.287352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05adb8-1206-45af-897d-3919f3d501bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.287969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:53.288709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05adb8-1206-45af-897d-3919f3d501bf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.305693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 488/990] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:30:53.318290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41a7b7-53c5-4654-8854-b3164d2506c3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.318585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41a7b7-53c5-4654-8854-b3164d2506c3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.318891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41a7b7-53c5-4654-8854-b3164d2506c3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.320973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 489/991] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:53.327182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.327480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.327780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.346709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:53.347937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:53.376248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 503/992] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:53.399780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.427500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 490/993] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:53.453234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.495177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.508253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.529877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b4f15-7067-4539-9e40-9058e8e63cf4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.567878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.568689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.568982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.569266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.569687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebc6b1-2f4c-4899-b7bd-7aa48f3974dc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.569962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.571339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:53.584954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:53.586069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:53.587077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 504/994] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:30:53.591895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.611885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.612683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.612969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.613243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] PATH_INFO: `/v3/services/image` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.613928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.615410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:53.628159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:53.634611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:53.635761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 491/995] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:53.638237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.698348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749f19-3c23-4aff-a1a2-760f5ad9a791 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.733457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.734171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.734463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.734788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.735510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.735825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.737260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.742146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 505/996] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:53.747649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.755038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.760661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c47da-3bb1-45ee-b215-8df098af606f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.762915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:53.803025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.803698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.803985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.804263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.804970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] RBAC: Adding query filter params (name=image) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.805281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.806684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.817405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 492/997] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:53.819114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.863516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158abf58-04e3-4551-995d-5ab1b7b18031 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.900400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.901076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.901371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.901693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.902441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.902759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.904158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.909385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 506/998] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:53.914826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:53.928675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab885ed-ad1b-45cb-b155-7a3bd458e7df None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:53.966234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:53.966911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:53.967213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:53.967476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:53.968196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:53.968531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:53.969951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:53.974705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 493/999] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:53.999365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:54.027365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d032dc-b0d9-4166-84b0-0b38480effb0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.065485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.066215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.066507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.066793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.067609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] RBAC: Adding query filter params (interface=public, service_id=0c55708bf902476ebaa08227574b713a, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:54.067935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.069332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd367b-03a7-4c96-a086-1479640544d7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.081119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 507/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:53 2022] GET /identity/v3/endpoints?service_id=0c55708bf902476ebaa08227574b713a&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:54.082163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:54.113637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:54.197107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e7bd1-c230-4985-8c5c-b4dab0a051b2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:54.231452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364aa6a-94bf-426b-91fa-466a87359cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:54.232114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364aa6a-94bf-426b-91fa-466a87359cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:54.232402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364aa6a-94bf-426b-91fa-466a87359cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:54.232683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364aa6a-94bf-426b-91fa-466a87359cb1 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:54.233375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364aa6a-94bf-426b-91fa-466a87359cb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:54.235351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364aa6a-94bf-426b-91fa-466a87359cb1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:54.249767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 494/1001] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:54 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:30:55.451662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7af2-4793-4211-aa25-f0238b39c24e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.452011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7af2-4793-4211-aa25-f0238b39c24e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.452285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7af2-4793-4211-aa25-f0238b39c24e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.454067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 508/1002] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:55.459418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.459862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.460042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.477321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:55.478528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:55.546040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 495/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:55.569965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7af2-4793-4211-aa25-f0238b39c24e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.675363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7af2-4793-4211-aa25-f0238b39c24e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:55.710496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:55.711283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.711577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.711859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.712526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:55.713992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:55.726906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:55.727920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:30:55.728797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:55.733118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 509/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:55.734646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.756777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:55.840377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c04e2d-da98-4b1d-b3b4-3c8b497061a3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:55.875874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:55.876549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:55.876841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:55.877125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:55.877869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:55.878188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:55.879624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:55.889156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 496/1005] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:55.890706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:55.896124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:55.997430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b3a1-c7a4-4f90-acfd-9e16a445494f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.032291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.032974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.033261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.033541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.034297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:56.034612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.035994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c349f7-a4aa-4e05-bcc0-b006532c70b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.041376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 510/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Jun 15 14:30:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:56.047335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.131984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0cc95-a63f-4ecc-90a2-ee7faba5223a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.132296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0cc95-a63f-4ecc-90a2-ee7faba5223a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.132579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0cc95-a63f-4ecc-90a2-ee7faba5223a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.133997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 511/1007] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:56.139091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974dbb3-bd31-4875-ae09-204067bd73df None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.139393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974dbb3-bd31-4875-ae09-204067bd73df None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.139683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974dbb3-bd31-4875-ae09-204067bd73df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.143746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.157696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1974dbb3-bd31-4875-ae09-204067bd73df None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:56.158908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1974dbb3-bd31-4875-ae09-204067bd73df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:56.165773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84680e-43e6-4e34-8171-c1abc0cd6e90 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.204070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.204742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.205038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.205317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.206115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.207473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.229716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.229917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 497/1008] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:56.241859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 512/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:56.265004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.380867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359fb7e-cd3e-4bbc-a757-1637692ee186 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.411726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18b90-266f-4ba8-8b31-32da3e666960 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.412121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18b90-266f-4ba8-8b31-32da3e666960 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.412487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18b90-266f-4ba8-8b31-32da3e666960 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.414566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 513/1010] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:30:56.419536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.419833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.420217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.421567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.422889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.423181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.423467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.424240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.425580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.440652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 498/1011] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:56 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:56.442136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:56.443335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:56.533710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 514/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:56 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:56.556226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.560515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.670892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d1dc4-7f31-49d2-a99a-22c0a3f6da95 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.710452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473234-633b-498c-9f96-55a31af07060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.711259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473234-633b-498c-9f96-55a31af07060 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.711550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473234-633b-498c-9f96-55a31af07060 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.711832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473234-633b-498c-9f96-55a31af07060 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.712497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70473234-633b-498c-9f96-55a31af07060 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.713890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70473234-633b-498c-9f96-55a31af07060 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-70473234-633b-498c-9f96-55a31af07060 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:56.726870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:56.727995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:56.733890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 499/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:56.736478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.843058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635000-8a0c-4f68-8237-3702e8964aae None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:56.863754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:56.879716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:56.880416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:56.880717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:56.880985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:56.881728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:56.882049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:56.883416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:56.888442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 515/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:56.895086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473234-633b-498c-9f96-55a31af07060 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:56.965528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:57.001941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473234-633b-498c-9f96-55a31af07060 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.036081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.036761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.037050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.037334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.038099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:57.038418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:57.039780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e16f6d-3a29-4e41-8f6b-38a0b219d659 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:57.049408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 500/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Jun 15 14:30:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:57.050956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:57.155095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b49dd-46d5-4f70-aff6-9f7a3c535528 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:57.190570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf9b29-c439-4417-a52d-bfdf3a9a4de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:57.191422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf9b29-c439-4417-a52d-bfdf3a9a4de8 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:57.191711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf9b29-c439-4417-a52d-bfdf3a9a4de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:57.191995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf9b29-c439-4417-a52d-bfdf3a9a4de8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:57.192746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf9b29-c439-4417-a52d-bfdf3a9a4de8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:57.194170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf9b29-c439-4417-a52d-bfdf3a9a4de8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:57.217745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 516/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:30:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:30:58.440770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faae4f-5c28-4ed0-aa6a-f21fc9cd9ada None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.441299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faae4f-5c28-4ed0-aa6a-f21fc9cd9ada None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.441418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faae4f-5c28-4ed0-aa6a-f21fc9cd9ada None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.442869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 501/1017] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:58.448698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06207a-690c-4e66-b053-0cc55c3d76e4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.449154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06207a-690c-4e66-b053-0cc55c3d76e4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.449276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06207a-690c-4e66-b053-0cc55c3d76e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.467550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c06207a-690c-4e66-b053-0cc55c3d76e4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:58.468747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c06207a-690c-4e66-b053-0cc55c3d76e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:58.505607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.553634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 517/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:30:58.578976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faae4f-5c28-4ed0-aa6a-f21fc9cd9ada None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.601482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ef01-7fee-434c-a6de-61f164a643cd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.601815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ef01-7fee-434c-a6de-61f164a643cd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.602104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ef01-7fee-434c-a6de-61f164a643cd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.603623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 518/1019] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:58.608225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.608520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.608802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.625780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:58.626979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:58.691553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faae4f-5c28-4ed0-aa6a-f21fc9cd9ada None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.700186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 519/1020] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:58.722491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.725189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.728115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.728918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.729206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.729488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.730689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.732122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.737851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 502/1021] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:58.740142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.821596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:58.830088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6803fb-c0c0-487b-a9da-bd1e6254efdc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.846281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b0836-9d76-4013-8260-783e105103a1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:58.865365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.866197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.866485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.866765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] PATH_INFO: `/v3/services/image` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.867429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.868799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:58.880563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:58.881786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:30:58.882690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:30:58.883449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:58.883539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 520/1022] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:58.885724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:58.886068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:58.886360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:58.887095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:58.887665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:58.888527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:58.918975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 503/1023] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:30:58 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:30:59.012525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4201f8-1e49-4d89-8f79-4fd397abb979 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.051275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.051967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.052257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.052540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.053262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.053576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.054995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.060814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 521/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:59.067215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.127407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.176925 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eaaa2f-cee5-4b0c-bb63-441de2c4751e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.214052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.214730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.215023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.215305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.216018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.216329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.217711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.227883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 504/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:59.229162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.256707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.337572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739915ea-f0f0-4aea-8609-df1868118ae4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.372967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.373634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.373961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.374244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.375043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] RBAC: Adding query filter params (interface=public, service_id=cb9918af639e49668870d988b106bb35, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.375962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.377334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f307-38bc-4ccc-b743-2c83ae50ca2b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.386714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 522/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/endpoints?service_id=cb9918af639e49668870d988b106bb35&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:30:59.388719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.473999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302539e-2fc9-4eb5-bc11-ef46154d19d7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.474535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302539e-2fc9-4eb5-bc11-ef46154d19d7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.474632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302539e-2fc9-4eb5-bc11-ef46154d19d7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.476072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 523/1027] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:30:59.480886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.481181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.481464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.499072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:30:59.500925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:30:59.512108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ff74-4198-4b9f-bdff-09f358bb1dd1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.529750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.551304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.552360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.552799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.553085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.553823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.555199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.573691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 505/1028] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:30:59.590179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 524/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:30:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:30:59.608544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.722376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8dc6a-4c88-4452-ba19-803f50874ef6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.763014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.763841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.764127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.764400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.776326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:30:59.785845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:30:59.786803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:30:59.787660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 506/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:30:59.793361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.813351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:30:59.896118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9b640-85da-4c0d-a4d7-86dbc1d4f11c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:30:59.929979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:30:59.930685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:30:59.930975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:30:59.931261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:30:59.932015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:30:59.932329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:30:59.933706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:30:59.946826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 525/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:30:59.948126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:30:59.992104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.052734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf245fd4-c950-44d4-bda3-de3bc8d64a72 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.093626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.095070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.095559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.096023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.108808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.110563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:00.128646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:00.129580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:00.130907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:00.130907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:00.130907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:00.130907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:00.134075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 507/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:00.135803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.224562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.244713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110cc47-d0db-41d7-aa71-1219c0725b7c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.278609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.279280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.279568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.279846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.280575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:00.280888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.282838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.299408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 526/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:00.300996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.393195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.402894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b1f5-3cb1-40e5-8d58-13451f1ce039 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.436802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.437603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.437939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.438220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.457816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.459314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:00.474933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:00.475894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:00.481887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 508/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:00.482518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.586221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80d7f5-1bd6-4fb0-8771-03fd62fac9af None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.621125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.622144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.622608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.623061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.624125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:00.624685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.627334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.636300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.651671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 527/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:00.653083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:00.760170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fbb20-fc77-48ca-9347-c4102ace29d4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:00.794303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:00.794958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:00.795246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:00.795528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:00.817590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:00.866200 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:00.867639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:00.969731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 509/1036] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:00.971671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.073277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa676c20-13fe-4245-9139-60541b9b1328 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.111148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.111982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.112270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.112558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.124189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.125934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b660a6b0-3f9e-476c-9338-808677ab8cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:01.145492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:01.146490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:01.147483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:01.147483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:01.147483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:01.147483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:01.149986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 528/1037] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:01.152315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.169538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf7b37-f6b5-4bd5-934f-db26b073a0bc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.169865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf7b37-f6b5-4bd5-934f-db26b073a0bc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.170133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf7b37-f6b5-4bd5-934f-db26b073a0bc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.171635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 529/1038] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:01.176395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58662401-c731-496c-84ee-bd968111d81e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.176689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58662401-c731-496c-84ee-bd968111d81e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.176976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58662401-c731-496c-84ee-bd968111d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.194695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-58662401-c731-496c-84ee-bd968111d81e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:01.196707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-58662401-c731-496c-84ee-bd968111d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:01.211718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.257505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.267610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c4c3-df4a-44fc-ad6a-86100b00aab0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.280876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 530/1039] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:01.304059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.304729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.305016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.305303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.306057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:01.306370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.306470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58662401-c731-496c-84ee-bd968111d81e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.307734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.324060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 510/1040] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.325745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.412680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58662401-c731-496c-84ee-bd968111d81e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.432927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf322e-e4fd-4b55-ba6b-ec61629747da None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.451497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.457045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.457893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.458184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.458470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.459674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.461125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.462813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 531/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.468167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.484538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.486245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.486537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.486840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.496181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.498533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:01.514343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:01.515275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:01.516229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 511/1042] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:01.521680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.522493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.579405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063e5a6-32c8-423d-828c-5711922ba22c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.617203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.618273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.618776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.619121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.619939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.621430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.637001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425a154-da48-4bb7-bda0-5a14276440ab None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.649840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 532/1043] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:01 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:01.685620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.686390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.686627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.686899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.687827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:01.688865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.690468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7eb06a-37c0-4675-b325-66df3aa52b2c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.705804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 512/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:01.711810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.742741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bff24-3f73-4b78-a4f3-c2a853764fce None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.743038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bff24-3f73-4b78-a4f3-c2a853764fce None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.743360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bff24-3f73-4b78-a4f3-c2a853764fce None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.744788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 513/1045] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:01.750957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.751372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.751561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.769592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:01.770863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:01.803611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.838845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47294a20-591c-4255-9305-8ff494184da9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:01.842422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:01.856187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 514/1046] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:01.878462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:01.878851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:01.879144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:01.879239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:01.879426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:01.880273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] RBAC: Adding query filter params (role.id=31445885c860449bb44473c2cae11fe3, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=f5953aa40eb54d399b593000f3c74596) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:01.880597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:01.882157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:01.887929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 533/1047] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/role_assignments?user.id=f5953aa40eb54d399b593000f3c74596&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=31445885c860449bb44473c2cae11fe3 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:01.999296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c0fb7-63d0-4b11-b807-75e007ea2c0c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.036460 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.037275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.037563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.037871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.038549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.039925 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:02.052276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:02.053345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:02.054264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 515/1048] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Jun 15 14:31:01 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:02.059396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.162364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae955f-abd9-4020-8fc0-617307cd5056 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.196262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.196941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.197231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.197514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.198257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:02.198573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.199989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.211704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 534/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:02.213220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.243051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:02.316216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282d0ea-4440-454d-a43e-364a5c88b2e5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.350552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.351312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.351673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.353576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.353576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:02.353576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.354589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3617c12-b945-41b6-a9e4-3e31ef5336a3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.359780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 516/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:02.365254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:02.399504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:02.473901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1c3d9-a72a-47e8-8e13-8cc398f9762f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:02.509152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12719161-6c54-4724-b781-0a878f4a3307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:02.509852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719161-6c54-4724-b781-0a878f4a3307 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:02.510141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719161-6c54-4724-b781-0a878f4a3307 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:02.510423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719161-6c54-4724-b781-0a878f4a3307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:02.511174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12719161-6c54-4724-b781-0a878f4a3307 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:02.512540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12719161-6c54-4724-b781-0a878f4a3307 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:02.533842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 535/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:31:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:03.888505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1053d-6b26-4742-aca4-43f69a44d4d7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.889030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1053d-6b26-4742-aca4-43f69a44d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.889135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1053d-6b26-4742-aca4-43f69a44d4d7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.890607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 517/1052] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:03.896507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891a925-53cf-4658-baab-837d21e00ed3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:03.896942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891a925-53cf-4658-baab-837d21e00ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:03.897116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891a925-53cf-4658-baab-837d21e00ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:03.914120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8891a925-53cf-4658-baab-837d21e00ed3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:03.915317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8891a925-53cf-4658-baab-837d21e00ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:03.994309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 536/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:04.018289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1053d-6b26-4742-aca4-43f69a44d4d7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.038826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039acb7-6679-49fd-9e29-9ed575e522de None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.039130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039acb7-6679-49fd-9e29-9ed575e522de None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.039405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039acb7-6679-49fd-9e29-9ed575e522de None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.041013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 537/1054] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:04.045654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.045973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.046257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.065037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:04.066259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:04.075730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.116381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.128773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1053d-6b26-4742-aca4-43f69a44d4d7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.144105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 538/1055] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:04.164761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.165651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.165974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.166268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.166554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.178272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:04.186174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:04.187192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:04.188764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:04.193122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 518/1056] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:04.194453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.276360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fb100-a6aa-49fa-be55-5b16b94426f8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.307153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954b4ab-b6fa-4013-b961-348bc40b17aa None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.312717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.313570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.313890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.314173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.324570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:04.333366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:04.334380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:04.335432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 539/1057] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:04.341071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.346848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.348240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.349035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.349334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.349647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.350477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.350794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.352166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.361488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 519/1058] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:04.367060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.383981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.463261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477bf48-87fe-49b9-8d95-04a97015c379 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.476210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d8785-90a9-4c49-9947-88d5a5b2b460 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.502479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.503396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.503829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.504293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.505194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.505678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.507063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.510562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.511438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.511724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.511999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] PATH_INFO: `/v3/users/7af4060383db4c65be8e256bb6e3ca7f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.515839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 540/1059] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:04.521135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.526232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] RBAC: Authorizing `identity:get_user(user_id=7af4060383db4c65be8e256bb6e3ca7f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.527675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.534424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 520/1060] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/users/7af4060383db4c65be8e256bb6e3ca7f => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:04.535927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.591611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.607346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.637291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5bc4d-6159-4604-b7a3-a0a638af057c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.644513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c61b5-9191-4e4f-a3df-431c83d3b110 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.675410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bc115-a855-439a-a68d-829026292628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.676196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bc115-a855-439a-a68d-829026292628 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.676496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bc115-a855-439a-a68d-829026292628 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.676772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bc115-a855-439a-a68d-829026292628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.683687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4979aa-1460-419e-af01-2cb7c34b2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.684966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4979aa-1460-419e-af01-2cb7c34b2670 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.685257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4979aa-1460-419e-af01-2cb7c34b2670 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.685536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4979aa-1460-419e-af01-2cb7c34b2670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.687725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bc115-a855-439a-a68d-829026292628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.689240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bc115-a855-439a-a68d-829026292628 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-445bc115-a855-439a-a68d-829026292628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:04.707368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:04.708349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:04.709279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:04.709279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:04.709279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:04.709279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:04.709279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 541/1061] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:04.714302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb502c-8056-4ab2-a15b-cf3502ea330c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.714593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb502c-8056-4ab2-a15b-cf3502ea330c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.714877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb502c-8056-4ab2-a15b-cf3502ea330c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.716168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 542/1062] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:04.716854 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb502c-8056-4ab2-a15b-cf3502ea330c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.755448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4979aa-1460-419e-af01-2cb7c34b2670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.756900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4979aa-1460-419e-af01-2cb7c34b2670 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.818119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb502c-8056-4ab2-a15b-cf3502ea330c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:04.826453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.835986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:04.854168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:04.854772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.855061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.855341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.856065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:04.856378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:04.857757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:04.859497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 521/1063] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:31:04.861155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:04.861452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:04.861759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:04.878683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 543/1064] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:04.878683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.878948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:04.878948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:04.944970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 522/1065] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:04.945837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:04.975497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3640c4-034e-4e91-8539-64add0478218 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.008471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.009329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.009616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.009926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.011111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] RBAC: Authorizing `identity:get_project(project_id=1038af0f8bcf4247a8479475e24fa8cc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.012538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.014257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 544/1066] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:05.015022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.059662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.060991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.062494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190002-186b-4ae6-9d18-60aa43d33bc5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.097070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.097912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.098199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.098490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.106098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.107525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.115343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3a28b-0e13-4a24-ac2a-569d1b0983e9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:05.121952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:05.122902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:05.123867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 523/1067] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:05.124962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.153817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.154476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.154769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.155055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.223775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.225233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.252621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c2e00-b6c2-426f-8b99-7e15cb337184 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.271154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.287661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.288313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.288594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.288869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.289722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=1038af0f8bcf4247a8479475e24fa8cc, user.id=7af4060383db4c65be8e256bb6e3ca7f) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:05.290040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.291626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.309697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 524/1068] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/role_assignments?user.id=7af4060383db4c65be8e256bb6e3ca7f&scope.project.id=1038af0f8bcf4247a8479475e24fa8cc&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:05.309697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.329349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 545/1069] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:05.335644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.341482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.430520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89069f-40d8-48a7-a05c-bd40f72ccf43 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.448076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef79a8-5591-49aa-a113-79985efe580f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.478386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.479414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.479903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.480330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.481346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:05.481926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.483688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.485328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.486174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.486462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.486741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.498272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.499715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.505583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 525/1070] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:05.507660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.511360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:05.524124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:05.525093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:05.526447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:05.526447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:05.526447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:05.526447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:05.530020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 546/1071] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:05.531637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.603090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.623213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cb9f5-6952-4806-849f-9f51ecfdd2f5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.641251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c545a-7221-4c6b-8651-f48251e8c9d6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.660843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.661789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.662082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.662362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.673988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.675437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.677202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.677910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.678201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.678483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.679211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:05.679526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.680880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.692638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 547/1072] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:05.695684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:05.696621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:05.697616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:05.697616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:05.697616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:05.697616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:05.697616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 526/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:05.704263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.774418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:05.821406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb734-cb08-438f-a30d-8124f77686a9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:05.858325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:05.859007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:05.859296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:05.859616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:05.860960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:05.861212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:05.862608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:05.876720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 548/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:05.882841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:05.994600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd213c-46b8-41d9-b74d-96c0a33d35bf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.030533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.031337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.032314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.032657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.034618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:06.040472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.041941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:06.056379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:06.057295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:06.058656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 527/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:06.063430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.161632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324529ff-b756-49c2-967b-c891bb421388 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.179056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:06.195833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.196491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.196777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.197050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.197864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:06.198175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.199616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ebe54-f246-4086-bd4c-dcdced07825d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.211990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 549/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:06.217919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:06.314825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929a4fb-873d-418d-8120-d08f3e3e0a6a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:06.348033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61a935-0d91-49f9-aeb4-8344f84c0458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:06.348915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a935-0d91-49f9-aeb4-8344f84c0458 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:06.349197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a935-0d91-49f9-aeb4-8344f84c0458 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:06.349473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a935-0d91-49f9-aeb4-8344f84c0458 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/f5953aa40eb54d399b593000f3c74596/roles/31445885c860449bb44473c2cae11fe3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:06.351764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61a935-0d91-49f9-aeb4-8344f84c0458 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=f5953aa40eb54d399b593000f3c74596, role_id=31445885c860449bb44473c2cae11fe3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:06.353175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61a935-0d91-49f9-aeb4-8344f84c0458 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:06.365898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 528/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:06 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/f5953aa40eb54d399b593000f3c74596/roles/31445885c860449bb44473c2cae11fe3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:07.417902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefacd7-0cbb-4f6a-aeda-10385cd9d872 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.418438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefacd7-0cbb-4f6a-aeda-10385cd9d872 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.418556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefacd7-0cbb-4f6a-aeda-10385cd9d872 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.419837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 550/1078] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:07.425382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.425920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.426021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.446540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:07.446540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:07.496301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:07.515480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 529/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:07.538389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefacd7-0cbb-4f6a-aeda-10385cd9d872 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.639886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefacd7-0cbb-4f6a-aeda-10385cd9d872 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.673907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.674645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.674934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.675213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.685495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:07.693142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:07.694332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:07.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 551/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:07.700442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.799477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2181de-a19d-459a-b62f-27c3d55e226f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.832986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.833653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.833962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.834241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.834989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:07.835299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:07.836656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:07.845338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 530/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:07.851061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:07.890286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:07.948903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7e8c0-3ef4-43fd-a107-2a9c4a01292d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:07.982518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:07.983391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:07.983678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:07.983958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] PATH_INFO: `/v3/users/7af4060383db4c65be8e256bb6e3ca7f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:07.985165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] RBAC: Authorizing `identity:get_user(user_id=7af4060383db4c65be8e256bb6e3ca7f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:07.986621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:07.988259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 552/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/users/7af4060383db4c65be8e256bb6e3ca7f => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:07.993957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:08.040453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.093306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27328d21-16d9-420a-a176-c2a1b0e02820 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.125904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4737e5-f6a2-4582-8498-4d5024212a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.126861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4737e5-f6a2-4582-8498-4d5024212a54 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.127153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4737e5-f6a2-4582-8498-4d5024212a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.127431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4737e5-f6a2-4582-8498-4d5024212a54 None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.128622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4737e5-f6a2-4582-8498-4d5024212a54 None admin] RBAC: Authorizing `identity:get_project(project_id=1038af0f8bcf4247a8479475e24fa8cc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:08.130082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4737e5-f6a2-4582-8498-4d5024212a54 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:08.131789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 531/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:07 2022] GET /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:08.137681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:08.237174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307955e5-3a56-4415-91eb-647a651d6630 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.244214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.270738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cba85e-6462-49e2-865f-aac6dbc5a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.271629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba85e-6462-49e2-865f-aac6dbc5a9d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.271916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba85e-6462-49e2-865f-aac6dbc5a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.272193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba85e-6462-49e2-865f-aac6dbc5a9d4 None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc/users/7af4060383db4c65be8e256bb6e3ca7f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.274162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cba85e-6462-49e2-865f-aac6dbc5a9d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1038af0f8bcf4247a8479475e24fa8cc, user_id=7af4060383db4c65be8e256bb6e3ca7f, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:08.275585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cba85e-6462-49e2-865f-aac6dbc5a9d4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:08.289245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 553/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:08 2022] PUT /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc/users/7af4060383db4c65be8e256bb6e3ca7f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:08.489944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e3c1-5263-46b4-b8e4-6ad514ac7a13 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.490439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e3c1-5263-46b4-b8e4-6ad514ac7a13 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.490439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e3c1-5263-46b4-b8e4-6ad514ac7a13 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.491801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 532/1085] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:08.497151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.497534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.497876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.515669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:08.517365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:08.583195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 554/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:08.607163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82e3c1-5263-46b4-b8e4-6ad514ac7a13 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:08.710633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82e3c1-5263-46b4-b8e4-6ad514ac7a13 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.724843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.746021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.746763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.747050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.747328 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.758096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:08.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:08.767794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:08.768656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 533/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Jun 15 14:31:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:08.774019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:08.866295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:08.873253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d0cc-4519-46a3-b0f7-caf3516139fd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:08.907505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:08.908174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:08.908463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:08.908743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:08.909509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:08.909861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:08.911233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:08.919169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 555/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:31:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:08.925151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.022376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad3711-aaba-4784-84aa-6230538e11ea None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:09.055087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:09.055886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.056176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.056455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.066960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:09.068398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:09.091709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:09.092992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:09.094293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:09.094293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:09.094293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:09.094293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:09.094293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 534/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:09.098544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.214550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6016-1584-418c-bf1d-43dc32245ba5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:09.247505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:09.248164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.248450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.248725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.249434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:09.249768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:09.251253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:09.263135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 556/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:09.268819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.296776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:09.369509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7d02-c296-466e-b421-44c07dadfc37 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:09.402710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:09.403497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.403777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.404047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.411556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:09.412976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:31:09.427391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:09.428386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:09.429364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 535/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:09.434315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.457042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:09.543653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8627a3-0161-401d-92ce-580e91e4cf4c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:09.577361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:09.577989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.578274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.578971 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.579987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:09.580304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:09.581759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:09.599150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 557/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:09.604794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.688396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:09.702952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc8f5e-afce-49f7-a006-06b6cdafa717 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:09.736873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:09.737520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:09.737839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:09.738121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:09.802932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:09.804373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:09.887719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:09.903083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 536/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:31:09.909702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:09.984871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.007544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40335a2f-2e4f-437b-85d1-9c47a2c11fcd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.041891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.042716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.043002 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.043281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.053417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.054899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:10.072921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:10.073996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:10.074931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:31:10.074931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:10.074931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:31:10.074931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:10.077815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 558/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:10.080422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.178140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be27347-1565-4eb3-927f-2cf9745d0a51 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.211072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.211738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.212026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.212308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.213043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:10.213356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.214832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.226413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 537/1095] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:10.232139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.330149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db52022-0578-45ec-a5a8-6d17db3d9899 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.356265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.365162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.366021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.366306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.366580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.374228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.376399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a24529b1-ffc8-4348-8a93-5a17a34c228a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:31:10.391645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:10.392724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:10.393621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 559/1096] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:10.398645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.422643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ba5f9-67c6-49b8-a2bf-f07bc7496a35 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.422939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ba5f9-67c6-49b8-a2bf-f07bc7496a35 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.423222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ba5f9-67c6-49b8-a2bf-f07bc7496a35 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.424626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 560/1097] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:10.429489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.429806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.430093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.447352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:10.448656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:10.450116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.505202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f49e83-000a-4d72-921e-a4425bbac906 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.517931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 561/1098] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:10.539137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.539807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.540096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.540375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.540814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.541171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:10.541486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.542952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.556061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 538/1099] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:10.561931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.642361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44280a83-7ca0-4441-8973-2631d6ae4303 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.659710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.664664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70248c-4322-4bea-ad5b-cfb302a2959c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.671805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:10.679890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.680640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.680932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.681220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.691325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:10.699200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:10.700254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:10.701127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:10.701906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.701906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.701906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.701906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.702201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 562/1100] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:31:10.702279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] RBAC: Adding query filter params (role.id=31445885c860449bb44473c2cae11fe3, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=f5953aa40eb54d399b593000f3c74596) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:10.702279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.703123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.706778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.709932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 539/1101] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/role_assignments?user.id=f5953aa40eb54d399b593000f3c74596&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=31445885c860449bb44473c2cae11fe3 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:10.815653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e97be7-57e7-4485-a73d-ddf926aab250 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:10.851252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:10.851937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:10.852223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:10.852499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:10.853279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:10.853595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:10.855151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:10.864905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 563/1102] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:31:10.871598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:10.975966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ab6d6-ec40-4a23-906f-db5b30c1252d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.009906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.010792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.011085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.011365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] PATH_INFO: `/v3/users/7af4060383db4c65be8e256bb6e3ca7f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.012558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] RBAC: Authorizing `identity:get_user(user_id=7af4060383db4c65be8e256bb6e3ca7f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.014012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.015799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 540/1103] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:10 2022] GET /identity/v3/users/7af4060383db4c65be8e256bb6e3ca7f => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.022571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.069272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.131802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de822a5-b78b-4210-b6ab-0fa4f824e9a1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.165863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.166531 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.166824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.167105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.237140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.238546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.282992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.342736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 564/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:11.349123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.390047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:11.454995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ce21-335e-4e3b-aaf8-3a612cebd135 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.489968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3600be-26e0-491f-89e8-76a08b9dd97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.490766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3600be-26e0-491f-89e8-76a08b9dd97b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.491052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3600be-26e0-491f-89e8-76a08b9dd97b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.491328 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3600be-26e0-491f-89e8-76a08b9dd97b None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.492511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3600be-26e0-491f-89e8-76a08b9dd97b None admin] RBAC: Authorizing `identity:get_project(project_id=1038af0f8bcf4247a8479475e24fa8cc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.493968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3600be-26e0-491f-89e8-76a08b9dd97b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.495674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 541/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:11.502016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:11.606027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b08aa-641b-42d8-80da-76bdad5c688e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:11.638519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:11.639170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:11.639462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:11.639739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:11.640571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=1038af0f8bcf4247a8479475e24fa8cc, user.id=7af4060383db4c65be8e256bb6e3ca7f) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:11.641467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:11.643038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025f62-c1c5-41dc-8d53-5b72e7edc9bb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:11.649021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 565/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:11 2022] GET /identity/v3/role_assignments?user.id=7af4060383db4c65be8e256bb6e3ca7f&scope.project.id=1038af0f8bcf4247a8479475e24fa8cc&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:13.786008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd3b31-d5c0-453a-b69b-f16411a1cd88 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.786544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd3b31-d5c0-453a-b69b-f16411a1cd88 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.786637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd3b31-d5c0-453a-b69b-f16411a1cd88 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.787947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 542/1107] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:13.793160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694e498-114e-4809-875c-709734d9cc17 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:13.793542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694e498-114e-4809-875c-709734d9cc17 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:13.793766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694e498-114e-4809-875c-709734d9cc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:13.799899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:13.810818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3694e498-114e-4809-875c-709734d9cc17 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:13.812001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3694e498-114e-4809-875c-709734d9cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:13.883887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 566/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:13.909877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd3b31-d5c0-453a-b69b-f16411a1cd88 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.007760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd3b31-d5c0-453a-b69b-f16411a1cd88 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.014391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:14.042857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050206b0-566f-4e6c-a50c-bfee2f5689fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.043579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050206b0-566f-4e6c-a50c-bfee2f5689fd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.043863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050206b0-566f-4e6c-a50c-bfee2f5689fd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.044141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050206b0-566f-4e6c-a50c-bfee2f5689fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.045302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050206b0-566f-4e6c-a50c-bfee2f5689fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.046762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050206b0-566f-4e6c-a50c-bfee2f5689fd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.048988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 543/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:14.054644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694e498-114e-4809-875c-709734d9cc17 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:14.151031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694e498-114e-4809-875c-709734d9cc17 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:14.183401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fc83d-a30e-4b8e-a815-f02e60f7dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:14.184067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc83d-a30e-4b8e-a815-f02e60f7dc90 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:14.184355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc83d-a30e-4b8e-a815-f02e60f7dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:14.184634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc83d-a30e-4b8e-a815-f02e60f7dc90 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:14.185344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fc83d-a30e-4b8e-a815-f02e60f7dc90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:14.186796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fc83d-a30e-4b8e-a815-f02e60f7dc90 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:14.193842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:14.208084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 567/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:14 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:16.374784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ea207-9ef2-4fd3-980a-299a3d191f94 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.375841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ea207-9ef2-4fd3-980a-299a3d191f94 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.375965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ea207-9ef2-4fd3-980a-299a3d191f94 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.377294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 544/1111] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:16.383335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.383847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.384142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.402329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:16.403518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:16.482147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 568/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:16.505696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ea207-9ef2-4fd3-980a-299a3d191f94 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:16.606456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ea207-9ef2-4fd3-980a-299a3d191f94 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:16.640020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:16.640899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.641829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.642122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] PATH_INFO: `/v3/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.643612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] RBAC: Authorizing `identity:get_role(role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:16.644986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:16.646399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 545/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:16 2022] GET /identity/v3/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:16.652259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:16.749112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f9b9e-c366-44da-bda5-bced0d1cfdb9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:16.768938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:16.783771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:16.784640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.784931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.785214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] PATH_INFO: `/v3/users/a62027b98c964b4b8dff77b606e1b666` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:16.799239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] RBAC: Authorizing `identity:get_user(user_id=a62027b98c964b4b8dff77b606e1b666)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:16.800683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:16.802363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 569/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:16 2022] GET /identity/v3/users/a62027b98c964b4b8dff77b606e1b666 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:16.808186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:16.852091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:16.907489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d28327-d29d-411e-9ade-843a4893249f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:16.941089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:16.941774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:16.942052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:16.942343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.008102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.009540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.050248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:17.106030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 546/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:17.112980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.210047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74a815-ced9-42e6-8e1b-a6db4351e39a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.244446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5b8eb-0916-4ce0-a1e0-a79058007240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.245305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b8eb-0916-4ce0-a1e0-a79058007240 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.245591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b8eb-0916-4ce0-a1e0-a79058007240 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.245895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b8eb-0916-4ce0-a1e0-a79058007240 None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.247069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5b8eb-0916-4ce0-a1e0-a79058007240 None admin] RBAC: Authorizing `identity:get_project(project_id=1038af0f8bcf4247a8479475e24fa8cc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.248489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5b8eb-0916-4ce0-a1e0-a79058007240 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.250212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 570/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:17.256524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:17.352441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c90fb-e572-49a7-8b7c-53cf76052e65 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:17.385252 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:17.385938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:17.386226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:17.386531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:17.387416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] RBAC: Adding query filter params (role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=1038af0f8bcf4247a8479475e24fa8cc, user.id=a62027b98c964b4b8dff77b606e1b666) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:17.387759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:17.389121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95676c-1721-41c4-b050-5b1b3a98fbde None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:17.394659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 547/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:17 2022] GET /identity/v3/role_assignments?user.id=a62027b98c964b4b8dff77b606e1b666&scope.project.id=1038af0f8bcf4247a8479475e24fa8cc&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:19.463856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9eb0f-005e-4caf-8670-9510c2f4443a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.464290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9eb0f-005e-4caf-8670-9510c2f4443a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.464430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9eb0f-005e-4caf-8670-9510c2f4443a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.465856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 571/1118] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:19.471768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.472166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.472340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.483242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.490005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:19.491194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:19.556083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 548/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:19.579097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9eb0f-005e-4caf-8670-9510c2f4443a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.626129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.678942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9eb0f-005e-4caf-8670-9510c2f4443a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.711739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.712524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.712810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.713090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] PATH_INFO: `/v3/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.714570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] RBAC: Authorizing `identity:get_role(role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.716001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.717348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 572/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:19.724065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.819504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fa3c9-41a6-4cff-a721-544217e47fd8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.853284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:19.854264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:19.854553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:19.854835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] PATH_INFO: `/v3/users/a62027b98c964b4b8dff77b606e1b666` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:19.856027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] RBAC: Authorizing `identity:get_user(user_id=a62027b98c964b4b8dff77b606e1b666)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:19.857450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:19.859104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 549/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/users/a62027b98c964b4b8dff77b606e1b666 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:19.864932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:19.963849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e6cc6-96fa-4690-89e3-7325b233e5b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:19.978474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:19.999734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04505b-9c76-42c3-90f5-4ebcee06acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.000825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04505b-9c76-42c3-90f5-4ebcee06acfa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.001120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04505b-9c76-42c3-90f5-4ebcee06acfa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.001403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04505b-9c76-42c3-90f5-4ebcee06acfa None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.002596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04505b-9c76-42c3-90f5-4ebcee06acfa None admin] RBAC: Authorizing `identity:get_project(project_id=1038af0f8bcf4247a8479475e24fa8cc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.004016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04505b-9c76-42c3-90f5-4ebcee06acfa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.005716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 573/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:19 2022] GET /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:20.011750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:20.015609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:20.111795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06daf-4852-4d56-932b-df5ca0a71b51 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:20.146329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50cc20-3e9a-4830-986c-37d9373bc841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:20.147207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50cc20-3e9a-4830-986c-37d9373bc841 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:20.147487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50cc20-3e9a-4830-986c-37d9373bc841 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:20.147758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50cc20-3e9a-4830-986c-37d9373bc841 None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc/users/a62027b98c964b4b8dff77b606e1b666/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:20.149939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50cc20-3e9a-4830-986c-37d9373bc841 None admin] RBAC: Authorizing `identity:create_grant(project_id=1038af0f8bcf4247a8479475e24fa8cc, user_id=a62027b98c964b4b8dff77b606e1b666, role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:20.151353 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50cc20-3e9a-4830-986c-37d9373bc841 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:20.164270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 550/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:20 2022] PUT /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc/users/a62027b98c964b4b8dff77b606e1b666/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:22.262065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0f00c-a030-4e58-80e5-2453074c2ce6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.262595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0f00c-a030-4e58-80e5-2453074c2ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.262709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0f00c-a030-4e58-80e5-2453074c2ce6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.264069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 574/1124] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:22.269352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.269819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.269943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.286559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:22.287732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:22.344724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.353747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 551/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:22.377380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0f00c-a030-4e58-80e5-2453074c2ce6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.476550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0f00c-a030-4e58-80e5-2453074c2ce6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.509203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.510090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.510544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.510650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] PATH_INFO: `/v3/roles/12e8b463d8c1431a8a351eb81e8be5ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.512167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] RBAC: Authorizing `identity:get_role(role_id=12e8b463d8c1431a8a351eb81e8be5ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.513475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.514876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 575/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/roles/12e8b463d8c1431a8a351eb81e8be5ac => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:22.520769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.617679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3945441-1aeb-4211-ab21-df06fd86937a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.650184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.651112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.651394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.651667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] PATH_INFO: `/v3/users/a62027b98c964b4b8dff77b606e1b666` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.652832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] RBAC: Authorizing `identity:get_user(user_id=a62027b98c964b4b8dff77b606e1b666)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.654280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.655901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 552/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/users/a62027b98c964b4b8dff77b606e1b666 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:22.662128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:22.710464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.759164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00272d89-bf06-4b53-9100-a50f111b17d5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:22.793362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:22.794167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:22.794448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:22.794724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:22.860653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:22.862139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:22.908824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:22.957007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 576/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:22.962943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:23.022135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:23.060395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8ce5-10d2-4aec-8970-d03439526765 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:23.093338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef38691-0df8-404b-8c4b-626de18238bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:23.094418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef38691-0df8-404b-8c4b-626de18238bd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:23.094707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef38691-0df8-404b-8c4b-626de18238bd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:23.094986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef38691-0df8-404b-8c4b-626de18238bd None admin] PATH_INFO: `/v3/projects/1038af0f8bcf4247a8479475e24fa8cc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:23.096170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef38691-0df8-404b-8c4b-626de18238bd None admin] RBAC: Authorizing `identity:get_project(project_id=1038af0f8bcf4247a8479475e24fa8cc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:23.097598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef38691-0df8-404b-8c4b-626de18238bd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:23.099323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 553/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:22 2022] GET /identity/v3/projects/1038af0f8bcf4247a8479475e24fa8cc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:23.105527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:23.200834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ba4b2-c1c8-4d37-86c2-7875de6596af None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:23.234379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:23.235668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:23.235951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:23.236228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:23.237062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] RBAC: Adding query filter params (role.id=12e8b463d8c1431a8a351eb81e8be5ac, scope.project.id=1038af0f8bcf4247a8479475e24fa8cc, user.id=a62027b98c964b4b8dff77b606e1b666) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:23.237379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:23.238832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec143d-0b9c-4ea0-b867-aeb00b18e513 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:23.244687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 577/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:23 2022] GET /identity/v3/role_assignments?user.id=a62027b98c964b4b8dff77b606e1b666&scope.project.id=1038af0f8bcf4247a8479475e24fa8cc&role.id=12e8b463d8c1431a8a351eb81e8be5ac => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:25.366754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5fb26-c606-49a4-a55b-5e6b056d4006 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.367284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5fb26-c606-49a4-a55b-5e6b056d4006 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.367374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5fb26-c606-49a4-a55b-5e6b056d4006 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.368689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 554/1131] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:25.374013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.374413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.374580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.392532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:25.393782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:25.403568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:25.461108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 578/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:25.484386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5fb26-c606-49a4-a55b-5e6b056d4006 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:25.585338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5fb26-c606-49a4-a55b-5e6b056d4006 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:25.612428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:25.618348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1fb2f-0079-4e26-a5b0-4f052a897f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:25.619078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1fb2f-0079-4e26-a5b0-4f052a897f56 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.619365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1fb2f-0079-4e26-a5b0-4f052a897f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.619643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1fb2f-0079-4e26-a5b0-4f052a897f56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.620815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1fb2f-0079-4e26-a5b0-4f052a897f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:25.622411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1fb2f-0079-4e26-a5b0-4f052a897f56 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:25.624018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 555/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:25.630339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:25.728475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec568b65-ad2e-4ff9-bdc1-21b70100e6d1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:25.763099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081b9e0-1cc9-4325-8d43-f94fbaa05dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:25.763755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081b9e0-1cc9-4325-8d43-f94fbaa05dbe None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:25.764039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081b9e0-1cc9-4325-8d43-f94fbaa05dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:25.764315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081b9e0-1cc9-4325-8d43-f94fbaa05dbe None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:25.765028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081b9e0-1cc9-4325-8d43-f94fbaa05dbe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:25.767125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081b9e0-1cc9-4325-8d43-f94fbaa05dbe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:25.780259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:25.793107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 579/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:31:25 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:27.942773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afe1e-3dc2-4261-a02a-d4bdeb8c686d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.943306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afe1e-3dc2-4261-a02a-d4bdeb8c686d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.943425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afe1e-3dc2-4261-a02a-d4bdeb8c686d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.944715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 556/1135] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:27.950018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c718ff-d727-4303-9273-bea0447af402 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:27.950393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c718ff-d727-4303-9273-bea0447af402 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:27.950581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c718ff-d727-4303-9273-bea0447af402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:27.967785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-80c718ff-d727-4303-9273-bea0447af402 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:27.968950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-80c718ff-d727-4303-9273-bea0447af402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:28.034252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 580/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:28.059392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73afe1e-3dc2-4261-a02a-d4bdeb8c686d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:28.159891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73afe1e-3dc2-4261-a02a-d4bdeb8c686d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:28.193019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a3011-e2d0-48a2-bfdd-df6706dac7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:28.193781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a3011-e2d0-48a2-bfdd-df6706dac7ba None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:28.194071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a3011-e2d0-48a2-bfdd-df6706dac7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:28.194353 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a3011-e2d0-48a2-bfdd-df6706dac7ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:28.195530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a3011-e2d0-48a2-bfdd-df6706dac7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:28.196953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a3011-e2d0-48a2-bfdd-df6706dac7ba None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:28.198591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 557/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:28.204843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c718ff-d727-4303-9273-bea0447af402 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:28.300762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c718ff-d727-4303-9273-bea0447af402 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:28.325481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:28.334840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b138d51-5079-4275-a42f-a0ce01ca84e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:28.335500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138d51-5079-4275-a42f-a0ce01ca84e3 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:28.335788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138d51-5079-4275-a42f-a0ce01ca84e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:28.336066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138d51-5079-4275-a42f-a0ce01ca84e3 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:28.336785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b138d51-5079-4275-a42f-a0ce01ca84e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:28.338235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b138d51-5079-4275-a42f-a0ce01ca84e3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:28.357327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 581/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:28 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:30.513639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711e9a-e5c7-41f6-9229-b2385d473910 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.514294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711e9a-e5c7-41f6-9229-b2385d473910 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.514294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711e9a-e5c7-41f6-9229-b2385d473910 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.515602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 558/1139] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:30.521301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.521689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.521903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.539942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:30.541142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:30.606486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 582/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:30.630325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea711e9a-e5c7-41f6-9229-b2385d473910 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.695827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.732526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea711e9a-e5c7-41f6-9229-b2385d473910 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.764905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.765724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.766044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.766323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.777349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:30.785578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:30.786712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:30.787765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 559/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:30.792745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:30.893693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfbded-c392-471d-9adc-8c1ad52c3445 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:30.906459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:30.928928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:30.929596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:30.929910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:30.930189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:30.930937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:30.931247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:30.932616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:30.940495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 583/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:30.946442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.043858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab899-7010-44b6-a3f2-0032bf343564 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.068135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.076792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.077655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.077970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.078252 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] PATH_INFO: `/v3/users/4822b36da2b14b91a4fd49d5508b172b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.091646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] RBAC: Authorizing `identity:get_user(user_id=4822b36da2b14b91a4fd49d5508b172b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.093076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.094883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 560/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:30 2022] GET /identity/v3/users/4822b36da2b14b91a4fd49d5508b172b => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:31.101207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.199251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8e3f-47fc-4cdf-9c88-9d223e31edfc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.233068 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.233745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.234035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.234317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.274630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.299201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.300571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.396629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 584/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:31.402589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.499492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd602b-2e3c-4601-ab68-28b23d4d9d69 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.534703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13aa7ef-3944-4a39-9e56-ebc502e5e163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.535567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aa7ef-3944-4a39-9e56-ebc502e5e163 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.535855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aa7ef-3944-4a39-9e56-ebc502e5e163 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.536133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aa7ef-3944-4a39-9e56-ebc502e5e163 None admin] PATH_INFO: `/v3/projects/6e4c75f991fe4fb5944d347b236f8e91` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.537315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13aa7ef-3944-4a39-9e56-ebc502e5e163 None admin] RBAC: Authorizing `identity:get_project(project_id=6e4c75f991fe4fb5944d347b236f8e91)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.538784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13aa7ef-3944-4a39-9e56-ebc502e5e163 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.540511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 561/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/projects/6e4c75f991fe4fb5944d347b236f8e91 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:31.546698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:31.624242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:31.649903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f950a04-1d0f-4a7d-8c04-2f5f02a184be None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:31.683139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:31.683791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:31.684081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:31.684363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:31.685198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=6e4c75f991fe4fb5944d347b236f8e91, user.id=4822b36da2b14b91a4fd49d5508b172b) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:31.685518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:31.686972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be776-71f0-453b-954a-6ff98ea42250 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:31.693015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 585/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:31 2022] GET /identity/v3/role_assignments?user.id=4822b36da2b14b91a4fd49d5508b172b&scope.project.id=6e4c75f991fe4fb5944d347b236f8e91&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:33.777714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b25a5-9755-4195-a6ce-4ac52e3caa75 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.778230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b25a5-9755-4195-a6ce-4ac52e3caa75 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.778376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b25a5-9755-4195-a6ce-4ac52e3caa75 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.779864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 562/1147] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:33.785988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:33.786412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:33.786562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:33.803579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:33.804760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:33.870022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 586/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:33.892326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b25a5-9755-4195-a6ce-4ac52e3caa75 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:33.946395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:33.994283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b25a5-9755-4195-a6ce-4ac52e3caa75 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.027271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.028076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.028336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.028618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.039167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:34.046899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:34.048013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:34.048897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 563/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:34.054392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.119240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.151650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f529f-05ad-4831-8a06-2b9f073b668e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.184587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.185243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.185526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.185835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.186582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:34.186888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.188689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.196447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 587/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.202305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.303705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cbe31-eb21-426d-b92f-1c1c8c11c95a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.316484 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.338237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.339129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.339421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.339705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] PATH_INFO: `/v3/users/4822b36da2b14b91a4fd49d5508b172b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.340986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] RBAC: Authorizing `identity:get_user(user_id=4822b36da2b14b91a4fd49d5508b172b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.342463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.344203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 564/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/users/4822b36da2b14b91a4fd49d5508b172b => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.350148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.445523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17a48-1ac0-42ff-9f77-1eb30be38054 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.474166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:34.480180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62f461-6692-49e4-ae0f-86a7afe1b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.481040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62f461-6692-49e4-ae0f-86a7afe1b228 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.481325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62f461-6692-49e4-ae0f-86a7afe1b228 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.481603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62f461-6692-49e4-ae0f-86a7afe1b228 None admin] PATH_INFO: `/v3/projects/6e4c75f991fe4fb5944d347b236f8e91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.482806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62f461-6692-49e4-ae0f-86a7afe1b228 None admin] RBAC: Authorizing `identity:get_project(project_id=6e4c75f991fe4fb5944d347b236f8e91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.484226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62f461-6692-49e4-ae0f-86a7afe1b228 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.486036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 588/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:34 2022] GET /identity/v3/projects/6e4c75f991fe4fb5944d347b236f8e91 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:34.491966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:34.589363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a3c42-bf5a-47be-a36c-f2ae34d47c41 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:34.621132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1f5ec-3447-4d76-ae6a-a07bd1ae21cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:34.622046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1f5ec-3447-4d76-ae6a-a07bd1ae21cd None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:34.622337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1f5ec-3447-4d76-ae6a-a07bd1ae21cd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:34.622621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1f5ec-3447-4d76-ae6a-a07bd1ae21cd None admin] PATH_INFO: `/v3/projects/6e4c75f991fe4fb5944d347b236f8e91/users/4822b36da2b14b91a4fd49d5508b172b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:34.624574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1f5ec-3447-4d76-ae6a-a07bd1ae21cd None admin] RBAC: Authorizing `identity:create_grant(project_id=6e4c75f991fe4fb5944d347b236f8e91, user_id=4822b36da2b14b91a4fd49d5508b172b, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:34.626763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1f5ec-3447-4d76-ae6a-a07bd1ae21cd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:34.640106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 565/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:34 2022] PUT /identity/v3/projects/6e4c75f991fe4fb5944d347b236f8e91/users/4822b36da2b14b91a4fd49d5508b172b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:36.727469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a21e6d-21fb-402c-b18b-40fbf00fef55 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.727974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a21e6d-21fb-402c-b18b-40fbf00fef55 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.727974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a21e6d-21fb-402c-b18b-40fbf00fef55 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.729334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 589/1154] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:31:36.734565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.735630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.735749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.752786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:36.754010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:36.787419 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:36.820778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 566/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:36.844630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a21e6d-21fb-402c-b18b-40fbf00fef55 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:36.945270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a21e6d-21fb-402c-b18b-40fbf00fef55 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:36.978523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:36.979363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:36.979649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:36.979930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:36.990368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:36.997993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:36.999099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:37.000150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 590/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:37.005078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.101556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2367d-3170-40ed-aa6d-69591f46df88 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.133919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.134584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.134871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.135153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.135905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:37.136219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.137581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.146059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 567/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:37.151848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.209330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.250510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63c88-2f87-4727-ba57-bdc2b43f8fe1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.283645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.284507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.284794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.285072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] PATH_INFO: `/v3/users/4822b36da2b14b91a4fd49d5508b172b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.286286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] RBAC: Authorizing `identity:get_user(user_id=4822b36da2b14b91a4fd49d5508b172b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.287720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.289333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 591/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/users/4822b36da2b14b91a4fd49d5508b172b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:37.295065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.326312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.394992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c95bba-0de2-45dd-b005-ff8dbfa68a29 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.426895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.427538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.427829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.428113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.493649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.495111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.524402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.590382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 568/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:37.596276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.693124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb7a0d-6e76-4595-8b70-40f9b88056d7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.717752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.726596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4b5da-1961-4986-91e0-4c6e92e447a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.727456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4b5da-1961-4986-91e0-4c6e92e447a5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.727746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4b5da-1961-4986-91e0-4c6e92e447a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.728299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4b5da-1961-4986-91e0-4c6e92e447a5 None admin] PATH_INFO: `/v3/projects/6e4c75f991fe4fb5944d347b236f8e91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.729204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4b5da-1961-4986-91e0-4c6e92e447a5 None admin] RBAC: Authorizing `identity:get_project(project_id=6e4c75f991fe4fb5944d347b236f8e91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.730685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4b5da-1961-4986-91e0-4c6e92e447a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.732379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 592/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/projects/6e4c75f991fe4fb5944d347b236f8e91 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:37.739031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:37.837651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d277bf-1581-4f3f-b88d-6a0ff76332e5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:37.866510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:37.871899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:37.872549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:37.872833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:37.873112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:37.873974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=6e4c75f991fe4fb5944d347b236f8e91, user.id=4822b36da2b14b91a4fd49d5508b172b) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:37.874289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:37.876788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839947ea-cee3-424b-8537-87cd276efa3e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:37.882468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 569/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:37 2022] GET /identity/v3/role_assignments?user.id=4822b36da2b14b91a4fd49d5508b172b&scope.project.id=6e4c75f991fe4fb5944d347b236f8e91&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:40.043025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fe57-056a-4601-8eab-27fb55233e1d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.043662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fe57-056a-4601-8eab-27fb55233e1d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.043662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fe57-056a-4601-8eab-27fb55233e1d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.044962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 593/1162] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:40.050561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.050967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.051125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.067546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:40.068711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:40.133885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 570/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:31:40.158050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258fe57-056a-4601-8eab-27fb55233e1d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.258519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258fe57-056a-4601-8eab-27fb55233e1d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.292380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.293014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.293350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.293577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.352364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.360677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.362154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.458960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 594/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:40.464766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.560271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3434-6719-462e-ae1d-dbce18cb3104 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.587042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.596304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.597154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.597446 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.597756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.604367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.605841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3ff27186-0512-4851-9209-0830ea6231d6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:40.619987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:40.621277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:40.622308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 571/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:40.627814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:40.726671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:40.735515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694f71d-823d-4c66-914c-391b285d8e58 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:40.774291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:40.775074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:40.775461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:40.775886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:40.776635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:40.777102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:40.778858 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca04ed6-ac62-4b2d-844b-ef22445821af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:40.787174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 595/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Jun 15 14:31:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:42.892964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3541b-1f7c-4a5a-be9e-4f2580faf6d7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.893527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3541b-1f7c-4a5a-be9e-4f2580faf6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.893527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3541b-1f7c-4a5a-be9e-4f2580faf6d7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.894940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 572/1167] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:42.900038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631d69d-437e-45e4-aff8-ada78cb19069 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:42.900478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631d69d-437e-45e4-aff8-ada78cb19069 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:42.900617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631d69d-437e-45e4-aff8-ada78cb19069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:42.917175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3631d69d-437e-45e4-aff8-ada78cb19069 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:42.918384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3631d69d-437e-45e4-aff8-ada78cb19069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:42.987432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 596/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:43.010722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3541b-1f7c-4a5a-be9e-4f2580faf6d7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:43.110678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3541b-1f7c-4a5a-be9e-4f2580faf6d7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:43.143177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b6883-592d-4441-85d1-3525b31521ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:43.143782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b6883-592d-4441-85d1-3525b31521ca None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:43.144072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b6883-592d-4441-85d1-3525b31521ca None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:43.144355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b6883-592d-4441-85d1-3525b31521ca None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:43.145108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b6883-592d-4441-85d1-3525b31521ca None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:43.146497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b6883-592d-4441-85d1-3525b31521ca None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:43.161159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 573/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Wed Jun 15 14:31:43 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:45.297706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c52d1f-c979-47fd-94ac-f2f9a6dbea57 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.298257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c52d1f-c979-47fd-94ac-f2f9a6dbea57 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.298257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c52d1f-c979-47fd-94ac-f2f9a6dbea57 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.299662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 597/1170] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:45.304978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.305358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.305547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.322192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:45.323376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:45.365054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.389369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 574/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:45.413798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c52d1f-c979-47fd-94ac-f2f9a6dbea57 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.499387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:45.512216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c52d1f-c979-47fd-94ac-f2f9a6dbea57 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.545476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1697-7571-4467-a160-ec7ee23107a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.546367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1697-7571-4467-a160-ec7ee23107a2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.546662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1697-7571-4467-a160-ec7ee23107a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.546939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1697-7571-4467-a160-ec7ee23107a2 None admin] PATH_INFO: `/v3/domains/f9f7087867df413ab151695f03a2fb03` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.557076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1697-7571-4467-a160-ec7ee23107a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f9f7087867df413ab151695f03a2fb03)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.558484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1697-7571-4467-a160-ec7ee23107a2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.560142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 598/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Wed Jun 15 14:31:45 2022] GET /identity/v3/domains/f9f7087867df413ab151695f03a2fb03 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:45.566512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:45.661194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c5e07-4d50-4034-8068-137a1e804da2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:45.693710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8a22c-b78d-483b-82b8-351f362af6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:45.694369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a22c-b78d-483b-82b8-351f362af6eb None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:45.694659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a22c-b78d-483b-82b8-351f362af6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:45.694942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a22c-b78d-483b-82b8-351f362af6eb None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:45.695664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a22c-b78d-483b-82b8-351f362af6eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:45.697078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a22c-b78d-483b-82b8-351f362af6eb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:45.723612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 575/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:31:45 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:31:47.879664 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1a798-8f98-4ea8-9a84-715a509bfed5 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.880296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1a798-8f98-4ea8-9a84-715a509bfed5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.880296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1a798-8f98-4ea8-9a84-715a509bfed5 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.881599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 599/1174] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:47.886960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:47.887398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:47.887532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:47.904618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:47.905851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:47.928557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:47.973074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 576/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:47.997430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1a798-8f98-4ea8-9a84-715a509bfed5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.108484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1a798-8f98-4ea8-9a84-715a509bfed5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.143042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8cb86-5adf-4fea-a05c-11f6637f07e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.143831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8cb86-5adf-4fea-a05c-11f6637f07e8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.144116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8cb86-5adf-4fea-a05c-11f6637f07e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.144390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8cb86-5adf-4fea-a05c-11f6637f07e8 None admin] PATH_INFO: `/v3/domains/f9f7087867df413ab151695f03a2fb03` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.145555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8cb86-5adf-4fea-a05c-11f6637f07e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f9f7087867df413ab151695f03a2fb03)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.147017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8cb86-5adf-4fea-a05c-11f6637f07e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.148636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 600/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Wed Jun 15 14:31:47 2022] GET /identity/v3/domains/f9f7087867df413ab151695f03a2fb03 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:48.154543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:48.262360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b77828-0f13-4380-9960-d3bf1291f3a4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:48.295564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0fbee-2a6e-4016-a2d0-084ac34996f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:48.296241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0fbee-2a6e-4016-a2d0-084ac34996f8 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:48.296528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0fbee-2a6e-4016-a2d0-084ac34996f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:48.296810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0fbee-2a6e-4016-a2d0-084ac34996f8 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:48.298086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0fbee-2a6e-4016-a2d0-084ac34996f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:48.299496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0fbee-2a6e-4016-a2d0-084ac34996f8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:48.319432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 577/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:31:48 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:31:50.474249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07e23-b8ab-4749-a3f9-5b029cc3c319 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.474713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07e23-b8ab-4749-a3f9-5b029cc3c319 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.474806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07e23-b8ab-4749-a3f9-5b029cc3c319 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.476186 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 601/1178] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:50.481387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.481807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.481987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.498738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:50.499914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:50.508938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.566122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 578/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:50.589605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da07e23-b8ab-4749-a3f9-5b029cc3c319 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.608665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:50.690184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da07e23-b8ab-4749-a3f9-5b029cc3c319 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.723532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.724261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.724549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.724829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.735356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:50.743088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:50.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:50.745279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 602/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:50.750572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.849955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811fca6-020f-4ee6-b3dc-f45fa445fa16 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:50.884203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:50.884909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:50.885260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:50.885478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:50.886296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:50.886629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:50.888071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:50.897134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 579/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:50.902884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:50.979169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.002669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adcaa6-0d64-40fe-b108-7150b3d9780d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.036010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.036866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.037148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.037420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] PATH_INFO: `/v3/users/62e2130e76f24013a1e3db77ef3c8579` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.051277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=62e2130e76f24013a1e3db77ef3c8579)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.052700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.054369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 603/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:50 2022] GET /identity/v3/users/62e2130e76f24013a1e3db77ef3c8579 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:51.061262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.070989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.158737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f845-bc26-4ceb-bbcd-2a3ff504bc6a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.192276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00945574-e419-4539-987c-6e4a82086123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.192921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00945574-e419-4539-987c-6e4a82086123 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.193232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00945574-e419-4539-987c-6e4a82086123 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.193511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00945574-e419-4539-987c-6e4a82086123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.259546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00945574-e419-4539-987c-6e4a82086123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.260905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00945574-e419-4539-987c-6e4a82086123 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.268504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.354528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 580/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:51.360941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.457217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d94da-5917-4d6d-9151-d65f9f55a9d1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.491239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d22573-66e1-4587-899f-60e57f08f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.492099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d22573-66e1-4587-899f-60e57f08f406 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.492386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d22573-66e1-4587-899f-60e57f08f406 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.492666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d22573-66e1-4587-899f-60e57f08f406 None admin] PATH_INFO: `/v3/projects/0f92cfc727b84515b2821f4f01f82948` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.493869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d22573-66e1-4587-899f-60e57f08f406 None admin] RBAC: Authorizing `identity:get_project(project_id=0f92cfc727b84515b2821f4f01f82948)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.495301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d22573-66e1-4587-899f-60e57f08f406 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.496993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 604/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/projects/0f92cfc727b84515b2821f4f01f82948 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:51.503571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00945574-e419-4539-987c-6e4a82086123 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:51.600104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00945574-e419-4539-987c-6e4a82086123 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:51.607256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:51.634003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:51.634652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:51.634944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:51.635226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:51.636064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=0f92cfc727b84515b2821f4f01f82948, user.id=62e2130e76f24013a1e3db77ef3c8579) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:51.636384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:51.637832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bd52e-90cc-4c05-8e2d-99669e4f4a33 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:51.643576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 581/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:51 2022] GET /identity/v3/role_assignments?user.id=62e2130e76f24013a1e3db77ef3c8579&scope.project.id=0f92cfc727b84515b2821f4f01f82948&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:53.753207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6350ab-3023-4c5b-85e5-3105f7a59e6e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.753743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6350ab-3023-4c5b-85e5-3105f7a59e6e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.753743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6350ab-3023-4c5b-85e5-3105f7a59e6e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.755078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 605/1186] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:53.760402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:53.760791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:53.760960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:53.778726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:53.779907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:53.845553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 582/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:53.868086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6350ab-3023-4c5b-85e5-3105f7a59e6e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:53.871007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:53.971847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6350ab-3023-4c5b-85e5-3105f7a59e6e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.004293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.005007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.005295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.005573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.015294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:54.023565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:54.024675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:31:54.025763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 606/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:54.031123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.124669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.128238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fecbc0-de78-4a42-98d9-e030f4cbdff0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.161645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.162338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.162628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.162910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.163661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:54.163976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.165340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.173825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 583/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:54.179511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.240052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.278576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e95b-fe30-4b6c-82d1-2df095668574 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.311003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.311864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.312152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.312428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] PATH_INFO: `/v3/users/62e2130e76f24013a1e3db77ef3c8579` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.314538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] RBAC: Authorizing `identity:get_user(user_id=62e2130e76f24013a1e3db77ef3c8579)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.315964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.317573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 607/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/users/62e2130e76f24013a1e3db77ef3c8579 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:54.323317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.420875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac06b-8f79-4f50-b313-f399db65758c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.454871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f25e-e9dd-483c-8209-76bc53759693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.455998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f25e-e9dd-483c-8209-76bc53759693 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.456283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f25e-e9dd-483c-8209-76bc53759693 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.456564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f25e-e9dd-483c-8209-76bc53759693 None admin] PATH_INFO: `/v3/projects/0f92cfc727b84515b2821f4f01f82948` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.457767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681f25e-e9dd-483c-8209-76bc53759693 None admin] RBAC: Authorizing `identity:get_project(project_id=0f92cfc727b84515b2821f4f01f82948)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.459192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681f25e-e9dd-483c-8209-76bc53759693 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.460878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 584/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:54 2022] GET /identity/v3/projects/0f92cfc727b84515b2821f4f01f82948 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:54.466721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:54.565101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff3b7f-d980-4e0d-b948-45fcd785487d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:54.592086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:54.598951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91591838-3255-4223-80e6-6486934cda66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:54.599827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91591838-3255-4223-80e6-6486934cda66 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:54.600112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91591838-3255-4223-80e6-6486934cda66 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:54.600385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91591838-3255-4223-80e6-6486934cda66 None admin] PATH_INFO: `/v3/projects/0f92cfc727b84515b2821f4f01f82948/users/62e2130e76f24013a1e3db77ef3c8579/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:54.602326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91591838-3255-4223-80e6-6486934cda66 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f92cfc727b84515b2821f4f01f82948, user_id=62e2130e76f24013a1e3db77ef3c8579, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:54.603747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91591838-3255-4223-80e6-6486934cda66 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:54.615824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 608/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:31:54 2022] PUT /identity/v3/projects/0f92cfc727b84515b2821f4f01f82948/users/62e2130e76f24013a1e3db77ef3c8579/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:31:56.740796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00530d-aca4-4810-b02b-9e33a601d2c9 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.741300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00530d-aca4-4810-b02b-9e33a601d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.741300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00530d-aca4-4810-b02b-9e33a601d2c9 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.742694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 585/1193] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:31:56.748043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.748467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.748612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.764995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:31:56.766808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:31:56.832011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 609/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:31:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:31:56.855154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00530d-aca4-4810-b02b-9e33a601d2c9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:56.885548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:56.955823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00530d-aca4-4810-b02b-9e33a601d2c9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:56.987568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:56.988346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:56.988632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:56.988912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:56.999110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:31:57.007368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:31:57.008495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:31:57.009420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 586/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:31:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:31:57.014582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.118524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159a65-2739-4588-89e3-7f802d45a2ce None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.137931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.152679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.153342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.153626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.153935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.154683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:57.154990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.156342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.164141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 610/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.169939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.259672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.269706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31bed-273f-49aa-9be2-9ae6e58bcb3d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.303588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.304451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.304743 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.305026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] PATH_INFO: `/v3/users/62e2130e76f24013a1e3db77ef3c8579` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.306247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] RBAC: Authorizing `identity:get_user(user_id=62e2130e76f24013a1e3db77ef3c8579)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.307669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.309301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 587/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/users/62e2130e76f24013a1e3db77ef3c8579 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.315630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.412587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ef318-8ace-43ab-9ecd-089ba7abd86f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.446768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.447429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.447719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.448009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.495490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.514250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.515641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.609908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 611/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:31:57.616482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.712922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92765-3d4d-4338-97a5-fe65d654cbf6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.747658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a201-83d2-4e73-bc3d-b906de500443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.748509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a201-83d2-4e73-bc3d-b906de500443 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.748792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a201-83d2-4e73-bc3d-b906de500443 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.749067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a201-83d2-4e73-bc3d-b906de500443 None admin] PATH_INFO: `/v3/projects/0f92cfc727b84515b2821f4f01f82948` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.750250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1a201-83d2-4e73-bc3d-b906de500443 None admin] RBAC: Authorizing `identity:get_project(project_id=0f92cfc727b84515b2821f4f01f82948)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.752150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1a201-83d2-4e73-bc3d-b906de500443 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.753842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 588/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/projects/0f92cfc727b84515b2821f4f01f82948 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:31:57.759805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:31:57.839711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:31:57.857944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946e02d-a647-4417-a0dd-68bf5f35d1bf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:31:57.891792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:31:57.892438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:31:57.892724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:31:57.892999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:31:57.893856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=0f92cfc727b84515b2821f4f01f82948, user.id=62e2130e76f24013a1e3db77ef3c8579) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:31:57.894170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:31:57.895580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bcfb0-a312-40b1-ab55-b1334038ff95 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:31:57.901217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 612/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:31:57 2022] GET /identity/v3/role_assignments?user.id=62e2130e76f24013a1e3db77ef3c8579&scope.project.id=0f92cfc727b84515b2821f4f01f82948&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:07.743662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:07.898665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=27 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:54.630250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875ce4b-0be8-4a61-8fb5-d4bd7e87a9fc None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:54.630857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875ce4b-0be8-4a61-8fb5-d4bd7e87a9fc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:54.630857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875ce4b-0be8-4a61-8fb5-d4bd7e87a9fc None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:54.632528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 589/1201] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:32:54.638181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:54.638698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:54.638798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:54.656326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:54.658059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:54.758370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 613/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:54.787769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875ce4b-0be8-4a61-8fb5-d4bd7e87a9fc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:54.931397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875ce4b-0be8-4a61-8fb5-d4bd7e87a9fc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:54.994066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:54.995127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:54.995542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:54.996003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:55.013231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:55.015635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:55.046372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:55.048872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:55.051465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:55.051465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:55.051465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:55.051465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:32:55.055749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 590/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:55.057694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:55.187821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5d169-7298-4e1d-a6ca-5fd9b6caef1f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:55.225873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:55.232550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:55.233354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:55.233593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:55.233934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:55.234735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:55.235080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:55.236686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db93a2-f789-436f-b3f1-a346c119079d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:55.254979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 614/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:55.256379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:55.332403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:55.393420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070df8-4d54-4402-a693-b3d9b3fe58d0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:55.442207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae7c38-193a-4c65-984e-634783e6f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:55.443413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7c38-193a-4c65-984e-634783e6f6ec None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:55.443413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7c38-193a-4c65-984e-634783e6f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:55.443628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7c38-193a-4c65-984e-634783e6f6ec None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:55.444358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae7c38-193a-4c65-984e-634783e6f6ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:55.445981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae7c38-193a-4c65-984e-634783e6f6ec None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:55.485825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 591/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Jun 15 14:32:55 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:32:57.896895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871fd8d-45e2-434c-bf11-1a281045e36c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:57.897479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871fd8d-45e2-434c-bf11-1a281045e36c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:57.897479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871fd8d-45e2-434c-bf11-1a281045e36c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:57.898910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 615/1206] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:32:57.904599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:57.905107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:57.905192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:57.924844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:32:57.926513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:32:58.001382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 592/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:32:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:32:58.025984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871fd8d-45e2-434c-bf11-1a281045e36c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.155943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871fd8d-45e2-434c-bf11-1a281045e36c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.197594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.198437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.198730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.199011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.211766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:32:58.222094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:58.223146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:32:58.224013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:32:58.225818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 616/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:58.230165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.273598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.344323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3614e0-c143-47ba-9c03-d7ace33d9c11 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.386336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.387074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.387343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.387664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.388321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:58.388666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.390028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.398812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 593/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:58.404656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.466802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.535564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99363e-4d08-4d40-a557-f6ef81df5d7b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.586201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.587560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.587878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.588189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.603821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.606045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:58.626723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:58.628249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:58.629190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:58.629190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:58.629190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:58.629190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:32:58.629190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 617/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:58.635152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.763098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.768388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee49a8-c07a-4abe-9a1c-6056c728bd9a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.808078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.808754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.809046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.809330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:58.810109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:58.810408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:58.811785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:58.827159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 594/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:58.833999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:58.954066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b2c5e-054e-40c6-a00c-fbe8024f2edf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:58.956863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:58.993512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:58.994361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:58.994647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:58.994923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.004672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:59.009316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:32:59.039256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:59.041453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:32:59.043459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 618/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:59.051108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:59.185112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea355d-29a4-49ca-ae9d-6b068c667558 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:59.233623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:59.234698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.235102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.236517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.237529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:32:59.238086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:59.240037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:59.262426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 595/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Jun 15 14:32:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:32:59.268535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:59.393554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcb4e7-38ff-465b-ad7c-9243b9849d7e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:59.439170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:59.440154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.440540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.440969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.480118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:59.524208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:59.525911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:59.639498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 619/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:32:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:32:59.646008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:59.672605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:59.763108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14f2c-685d-42c1-a617-37bc86172baf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:32:59.800969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:32:59.801821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:32:59.802111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:32:59.802390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:32:59.813475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:32:59.815458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:32:59.835098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:32:59.836099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:32:59.837013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:32:59.837013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:32:59.837013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:32:59.837013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:32:59.841720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 596/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:32:59.842125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:32:59.916769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:32:59.960757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6176-18af-4be4-8267-c942360436c0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:00.000620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:00.001908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:00.002272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:00.002556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:00.003300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:00.003649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:00.005327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:00.019049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 620/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:33:00.024566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:00.118606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:00.153077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f6994-9545-4b7d-8c82-2483672f9503 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:00.195291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:00.196121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:00.196477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:00.196835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:00.207494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:00.209293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:33:00.232513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:00.233792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:00.234942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 597/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:00.241724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:00.377936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413dbd7-3603-4354-a446-21de3bcfbe9b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:00.405008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:00.419967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:00.420747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:00.421550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:00.421871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:00.422681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:00.423005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:00.425228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef55281-79a2-42cc-b693-f1284ffde99b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:00.439592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 621/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:00.449485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:00.579332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d9e7a-e7a5-4d2c-86db-69cb07afd554 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:00.617356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:00.624345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:00.625017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:00.625312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:00.625717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:00.627320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=fddfa14cf721489595e79ffaafef6322) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:00.627629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:00.631608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cff96-1c96-46e4-bcc0-d0c15b53fddb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:00.638964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 598/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:00 2022] GET /identity/v3/role_assignments?user.id=fddfa14cf721489595e79ffaafef6322&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:03.127653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdb2f9-dd2b-4eca-958f-58b9c27217ea None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:03.128188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdb2f9-dd2b-4eca-958f-58b9c27217ea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:03.128302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdb2f9-dd2b-4eca-958f-58b9c27217ea None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:03.129777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 622/1220] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:03.135523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:03.136065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:03.136177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:03.153115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:03.154345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:03.240187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 599/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:03.266502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdb2f9-dd2b-4eca-958f-58b9c27217ea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:03.380244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdb2f9-dd2b-4eca-958f-58b9c27217ea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:03.415137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:03.415944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:03.416234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:03.416517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:03.426904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:03.434697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:03.435898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:03.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 623/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:33:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:03.443025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:03.578095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbfbdd-8d4d-4463-813b-60c8b73e31b6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:03.614303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:03.629462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:03.630741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:03.631151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:03.631670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:03.632673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:03.633173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:03.635222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:03.645930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 600/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:33:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:03.651733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:03.702426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:03.767640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac0a5-7aa3-4897-820b-212c24755e4b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:03.811592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:03.812557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:03.812999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:03.813382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:03.826362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:03.828529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:03.852373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:03.853548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:03.854652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:03.854652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:03.854652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:03.854652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:03.854652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 624/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:03.860005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:03.978085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cafc3-b7c5-4051-a0ec-a7cbc391bcf3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:04.015282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:04.015958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:04.016246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:04.016566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:04.017485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:04.017884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:04.019355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:04.032287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 601/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:04.038368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:04.147830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:04.155901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f7a29-cca3-4bc7-9a80-d2da837ab90a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:04.192643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:04.193461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:04.193778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:04.194059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:04.201963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:04.203405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:33:04.219598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:04.220795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:04.222050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 625/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:04.227241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:04.285041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:04.351648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211b11-11cf-4ecb-9317-0793047b696d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:04.390270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:04.390967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:04.391297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:04.391618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:04.392576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:04.392933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:04.394494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:04.416140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 602/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Jun 15 14:33:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:04.422601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:04.538093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baa023-2388-42c7-a490-c0dfa2ee962f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:04.578676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:04.579604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:04.579895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:04.580641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:04.593443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:04.595714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:04.615398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:04.616502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:04.617794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:04.617794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:04.617794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:04.617794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:04.617794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 626/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:04.622633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:04.732532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0169bb-29b8-486e-939e-05ad2bc40de6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:04.734913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:04.768399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:04.769139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:04.769464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:04.769808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:04.770562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:04.770879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:04.772741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:04.786322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 603/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:04.792221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:04.803174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:04.911786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ef75d-bb88-4f82-9b11-f6ceb3bf26f7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:04.949339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:04.950280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:04.950617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:04.950928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:04.958783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:04.960380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:33:04.975562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:04.976519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:04.977422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 627/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:04.983039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.093279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10cec8-95b7-457c-9b8d-1a991396e996 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:05.130479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:05.131154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.131444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.131757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.132790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:05.133149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:05.135010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20ba9a-67a6-4537-b6d5-611066bc35a7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:05.148364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 604/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:05.154581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:05.233314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:05.276886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dadd2a-601b-428e-805e-7ea58dfac9d4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:05.316021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac581b6-19ad-4b99-83ed-ddb134b9bbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:05.316933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac581b6-19ad-4b99-83ed-ddb134b9bbe1 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:05.317225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac581b6-19ad-4b99-83ed-ddb134b9bbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:05.317526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac581b6-19ad-4b99-83ed-ddb134b9bbe1 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/fddfa14cf721489595e79ffaafef6322/roles/903cbcc96ea745dfa8e911069499b16e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:05.333503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac581b6-19ad-4b99-83ed-ddb134b9bbe1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=fddfa14cf721489595e79ffaafef6322, role_id=903cbcc96ea745dfa8e911069499b16e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:05.335205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac581b6-19ad-4b99-83ed-ddb134b9bbe1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:05.349494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 628/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:33:05 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/fddfa14cf721489595e79ffaafef6322/roles/903cbcc96ea745dfa8e911069499b16e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:33:07.930177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03cbf-b343-4878-8896-48426db4db15 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:07.930750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03cbf-b343-4878-8896-48426db4db15 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:07.930750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03cbf-b343-4878-8896-48426db4db15 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:07.932319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 605/1233] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:07.937467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ff727-4a93-4801-bd08-806269f6abba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:07.937921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ff727-4a93-4801-bd08-806269f6abba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:07.938099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ff727-4a93-4801-bd08-806269f6abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:07.955054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-950ff727-4a93-4801-bd08-806269f6abba None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:07.956271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-950ff727-4a93-4801-bd08-806269f6abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:08.028627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 629/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:08.054623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe03cbf-b343-4878-8896-48426db4db15 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.084374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.179907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe03cbf-b343-4878-8896-48426db4db15 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.217913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.218659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.218928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.219213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.231136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:08.239703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:08.241000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:08.242275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 606/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:08.247848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ff727-4a93-4801-bd08-806269f6abba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.304041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.363503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ff727-4a93-4801-bd08-806269f6abba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.397902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.398570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.398866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.399171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.399909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:08.400221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.401600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:08.410997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 630/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:08.416778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.513241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.531872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed0111-c0d7-4b51-88ae-20a0a927a6bf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.570412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.571433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.571719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.572003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.585095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.586955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:08.607418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:08.608542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:08.609891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:08.609891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:08.609891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:08.609891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:08.609891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 607/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:08.614482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.725795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba5af5-bb54-4c1e-ab02-8c7e0434ffa3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.736137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.765490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.766116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.766406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.766693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.767983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:08.768303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.769700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:08.781268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 631/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:08.787421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:08.913257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d29f-d235-4309-8616-ecb192d0c905 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:08.953417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:08.954474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:08.956046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:08.956411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:08.961449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:08.969017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:08.971625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:33:08.990308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:08.991469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:08.992442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 608/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:33:08.998748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:09.117963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb1fc9-33e8-4e56-a475-b226e996ed6b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:09.159816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:09.160467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:09.160758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:09.161042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:09.161938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:09.162276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:09.163722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.182537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 632/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Jun 15 14:33:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:09.190516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:09.328024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c770d8-c44b-460a-8f68-a7f2fd8c75d5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:09.365436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:09.366143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:09.366439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:09.366722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:09.429571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:09.436495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:09.437977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.550118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 609/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:09.553230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:09.581458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:09.666233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce9675-6d65-459b-b673-7ddd31c1de85 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:09.704335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:09.705718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:09.706260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:09.706512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:09.717730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:09.719177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:09.739680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:09.740767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:09.741766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:09.741766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:09.741766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:09.741766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:09.741766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 633/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:33:09.746971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:09.854613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:09.862149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87da93b-372d-4243-95a5-af14c35c18d9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:09.901326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:09.902039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:09.902358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:09.902669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:09.903351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:09.903696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:09.905036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:09.917617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 610/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:09.924570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:10.020070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:10.043033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a1964-8493-4076-90a4-fed6111e0e84 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:10.082213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:10.083015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:10.083307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:10.083824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:10.090697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:10.092167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:33:10.107794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:10.109081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:10.110254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 634/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:33:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:10.115991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:10.227898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5ef5c-8445-4dea-a454-e66ec5002079 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:10.267707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:10.269027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:10.269351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:10.269687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:10.270733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:10.271025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:10.272672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35d6d-afd5-4a82-8f41-a965083b019e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:10.289352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 611/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:10.295558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:10.415625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020e32-fefc-4528-83bd-12df1de901cf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:10.454460 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:10.456575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:10.457040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:10.457423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:10.459329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] RBAC: Adding query filter params (role.id=903cbcc96ea745dfa8e911069499b16e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=fddfa14cf721489595e79ffaafef6322) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:10.460014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:10.462684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dacd-c0d1-4212-aaa6-56c8da7b01e7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:10.473876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 635/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:10 2022] GET /identity/v3/role_assignments?user.id=fddfa14cf721489595e79ffaafef6322&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=903cbcc96ea745dfa8e911069499b16e => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:12.778138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b164-55ce-48f5-b81c-36063b8a5f74 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:12.778775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b164-55ce-48f5-b81c-36063b8a5f74 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:12.778775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b164-55ce-48f5-b81c-36063b8a5f74 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:12.780261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 612/1247] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:12.786247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:12.786726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:12.786997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:12.792678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:12.807702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:12.809108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:12.881226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 636/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:33:12.904493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493b164-55ce-48f5-b81c-36063b8a5f74 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:12.923194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:13.011784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493b164-55ce-48f5-b81c-36063b8a5f74 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:13.048699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:13.049428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:13.049808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:13.050041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:13.060369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:13.068036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:13.069078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:13.070341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 613/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:13.075529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:13.203862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc157e0f-8a5a-45c7-807b-0304ac65e5e6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:13.243016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:13.248435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:13.249406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:13.249730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:13.250073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:13.251128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:13.251517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:13.253542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:13.264941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 637/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:13.272084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:13.363855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:13.390773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24808ec-301b-4808-991e-b7b6000f9bfd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:13.428708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:13.429592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:13.429930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:13.430281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:13.443020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:13.444736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:13.467319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:13.468445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:13.470831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:13.470831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:13.470831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:13.470831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:13.470831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 614/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:13.474488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:13.587154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa288a-d4d2-402f-b61a-27368d148d86 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:13.623581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:13.624155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:13.624438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:13.624718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:13.625442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:13.625788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:13.628313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:13.640259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 638/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:13.647185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:13.768894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a6e6-825c-48b9-9ba6-f79f131622fc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:13.797644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:13.807010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:13.807840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:13.808123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:13.808398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:13.818112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:13.819569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:33:13.835423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:13.836684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:13.837937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 615/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:33:13.842656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:13.902679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:13.952024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36448745-a29c-4ac7-aff1-48c5b78f3553 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:13.988863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:13.989741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:13.990032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:13.990309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:13.991183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:13.991500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:13.992963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.012617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 639/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Jun 15 14:33:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:14.018828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:14.124884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b64c9-9b8a-4665-a4de-53bd33042a2a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:14.160524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:14.161098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.161392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.161687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.219382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:14.236921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:14.238645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.348126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 616/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:14.354279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:14.461863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564c26e-5924-40aa-a7be-7e19877b6d28 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:14.475709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:14.496889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:14.497703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.497999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.498279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.508389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:14.509858 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:14.527423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:14.528470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:14.529594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:14.529594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:14.529594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:14.529594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:14.529594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 640/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:14.535324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:14.619278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:14.631247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5180c8-63bf-4ad0-8652-536b72c79def None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:14.665041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:14.665724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.666016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.666299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.667027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:14.667350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:14.668756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.679797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 617/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:33:14.685416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:14.781080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1878f47-61cd-4e94-a4c3-5f82005ce5e9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:14.816568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:14.817349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.817635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.817944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.824409 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:14.825875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:33:14.840299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:14.841495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:14.842653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 641/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:14.850306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:14.950363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839f76-d8ad-4435-81fe-6d9ac4b9a541 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:14.983717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:14.984373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:14.984659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:14.984941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:14.985765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:14.986195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:14.987600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9dbf7-7f0d-430e-b702-cb7bfdd08ae8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:14.999836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 618/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:15.007319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:15.032894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:15.105542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557998d-92f6-4fe2-9a7f-268b65a05030 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:15.138130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:15.138783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:15.139074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:15.139357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:15.140194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=fddfa14cf721489595e79ffaafef6322) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:15.140512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:15.141961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aee937-d608-44a7-bae5-aec0c10a840e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:15.148035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 642/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:15 2022] GET /identity/v3/role_assignments?user.id=fddfa14cf721489595e79ffaafef6322&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:17.284854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8660d-e6bd-46a2-8152-3d582985e9be None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:17.285472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8660d-e6bd-46a2-8152-3d582985e9be None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:17.285603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8660d-e6bd-46a2-8152-3d582985e9be None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:17.287169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 619/1261] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:17.292687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:17.293286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:17.293388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:17.311446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:17.312630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:17.377577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 643/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:17.400206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8660d-e6bd-46a2-8152-3d582985e9be None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:17.412301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:17.504394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8660d-e6bd-46a2-8152-3d582985e9be None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:17.537639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:17.538395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:17.538681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:17.538960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:17.548825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:17.556474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:17.557603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:17.558531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 620/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:17.563753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:17.582002 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:17.660902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0938-d891-4f3f-b709-2bbb6ae05e95 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:17.695387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:17.696059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:17.696349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:17.696630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:17.697383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:17.697720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:17.699086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:17.707791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 644/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:17.714356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:17.782802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:17.817349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df043-6e2c-4e57-a089-a9ef9630bde9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:17.860287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:17.861774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:17.862113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:17.862488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:17.879838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:17.882470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:17.905588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:17.906572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:17.907516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:17.907516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:17.907516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:17.907516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:17.907742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 621/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:17.914504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:17.998603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:18.018910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006127db-da03-4cd1-9f25-e28e6f8f2936 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:18.056216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:18.057256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:18.057559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:18.057876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:18.058618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:18.058937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:18.060310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:18.072730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 645/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:18.078531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:18.187313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec75f-05e1-446c-8210-80c87de9b5a0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:18.209622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:18.227578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:18.228773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:18.229173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:18.229460 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:18.239199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:18.240790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:33:18.256458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:18.257625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:18.259002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 622/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:18.265983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:18.377047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c95b59-91dd-463f-9069-db5f6b10b8cb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:18.414497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:18.415202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:18.415495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:18.415778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:18.416639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:18.416985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:18.418486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:18.431053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:18.441593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 646/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Jun 15 14:33:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:18.447383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:18.553807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5555818-8a8b-42cf-8d88-40866931c62c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:18.588831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:18.589631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:18.589948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:18.590234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:18.601889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:18.603345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:18.620395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:18.621511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:18.622521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:18.622521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:18.622521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:18.622521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:18.622521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 623/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:18.627981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:18.726462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8339f-b40b-4e57-8251-3b442ea7175c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:18.759377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:18.760058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:18.760347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:18.760626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:18.761349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:18.761687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:18.763050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:18.774000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 647/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:18.779632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:18.812351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:18.879272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfca858-27c4-4668-aa30-ad4324cb6cb1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:18.911659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:18.912447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:18.912735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:18.913017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:18.919502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:18.920929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:33:18.935405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:18.936444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:18.937414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 624/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:18.942545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:19.001182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:19.042488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ea451-94a4-42a4-ab4c-fc20871355ed None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:19.074375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:19.075035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:19.075323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:19.075605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:19.076397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:19.076714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:19.078177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9dce3-bfef-4cd4-b8da-49bdf2d43eec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:19.090784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 648/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:19.097512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:19.186052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:19.195418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e3763-1a06-4d53-83cc-d8cc12ceaa56 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:19.234064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7244b-d547-4264-936e-88507e2580b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:19.235225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7244b-d547-4264-936e-88507e2580b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:19.235548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7244b-d547-4264-936e-88507e2580b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:19.235928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7244b-d547-4264-936e-88507e2580b5 None admin] PATH_INFO: `/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/fddfa14cf721489595e79ffaafef6322/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:19.238617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7244b-d547-4264-936e-88507e2580b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c210b380e73f4922a6e6c1ecc8e4bc6b, user_id=fddfa14cf721489595e79ffaafef6322, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:19.240083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7244b-d547-4264-936e-88507e2580b5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:19.254672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 625/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Jun 15 14:33:19 2022] PUT /identity/v3/projects/c210b380e73f4922a6e6c1ecc8e4bc6b/users/fddfa14cf721489595e79ffaafef6322/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:33:21.372724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe809f0-fd69-46c2-9c20-e033a4cfc291 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:21.373303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe809f0-fd69-46c2-9c20-e033a4cfc291 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:21.373303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe809f0-fd69-46c2-9c20-e033a4cfc291 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:21.374685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 649/1274] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:21.380308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:21.380715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:21.380869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:21.397125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:21.398336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:21.465179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 626/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:21.488869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe809f0-fd69-46c2-9c20-e033a4cfc291 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:21.589901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe809f0-fd69-46c2-9c20-e033a4cfc291 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:21.606909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:21.624219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05142b-9077-4107-be28-f1d150240685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:21.624943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05142b-9077-4107-be28-f1d150240685 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:21.625226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05142b-9077-4107-be28-f1d150240685 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:21.625501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05142b-9077-4107-be28-f1d150240685 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:21.635696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05142b-9077-4107-be28-f1d150240685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-db05142b-9077-4107-be28-f1d150240685 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:33:21.643781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:21.644876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:21.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 650/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Jun 15 14:33:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:21.650878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:21.730537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:21.749819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e987cd-69ec-4830-b317-9443867975d7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:21.782420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:21.783577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:21.783866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:21.784238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:21.784991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:21.785308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:21.786707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:21.794537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 627/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Jun 15 14:33:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:21.800885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05142b-9077-4107-be28-f1d150240685 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:21.897787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05142b-9077-4107-be28-f1d150240685 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:21.930617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:21.931399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:21.931688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:21.931966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:21.942327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:21.943758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:21.961066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:21.962140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:21.967896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:21.967896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:21.967896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:21.967896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:21.967896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 651/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:21.968261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.065673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9eb5-1b3a-4c29-9116-fbec5be8bf03 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.099505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.100152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.100439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.100722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.101440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:22.101780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.103136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.115314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 628/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:33:22.121128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.151621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.224171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09871942-e5c3-4020-8fd7-d13933668c3f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.260074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.260908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.261167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.261437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.269947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.271394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:33:22.286419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:22.287548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:22.292950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 652/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:22.293734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.297461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.388949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77e01-35ab-4e91-a32d-10ca4da3b16c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.423259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.423885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.424167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.424444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.425274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:22.425584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.432348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.456214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 629/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:22.461936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.551625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.561518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f94198-50a9-48f4-94d5-12b93e1be4f2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.594763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.595407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.595695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.595976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.661632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.663103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.750232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.761653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 653/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:33:22.767724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:22.827173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:22.864934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf403bb-5f7d-43f7-97ae-6747a7a24b40 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:22.897650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:22.898462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:22.898754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:22.899033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:22.908796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:22.910272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:33:22.927961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:22.929050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:22.930044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:33:22.930044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:33:22.930044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:33:22.930044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:22.930044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 630/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:22.934893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:23.031522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a87c-db0f-4858-a97c-ba2259e77325 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:23.065745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:23.066400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:23.066682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:23.066956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:23.067674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:23.067983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:23.069326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:23.080183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 654/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Jun 15 14:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:23.086096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:23.182761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39835ba7-4eb1-46d1-bd5d-3d5b3152e79e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:23.196578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:23.217872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:23.219007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:23.219361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:23.219761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:23.228097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:23.229540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:33:23.243732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:23.244809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Jun 15 14:33:23.246148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 631/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Jun 15 14:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:23.250710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:23.299910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:23.348208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ec4a0-da65-444a-8bdb-47bb47fd5b6b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:23.381802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:23.382469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:23.382761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:23.383046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:23.383842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:23.384162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:23.385593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b3260-46fc-413c-830e-5bfc0732937a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:23.397829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 655/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Jun 15 14:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:23.403913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:23.504984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615667a-60ab-449c-b38a-74cdbb8e8d16 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:23.538675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:23.539324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:23.539612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:23.539891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:23.540719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] RBAC: Adding query filter params (role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e, scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b, user.id=fddfa14cf721489595e79ffaafef6322) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:23.541035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:23.542469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cfc2b-f52a-41fa-84e2-77c397acec8b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:23.548119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 632/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Jun 15 14:33:23 2022] GET /identity/v3/role_assignments?user.id=fddfa14cf721489595e79ffaafef6322&scope.project.id=c210b380e73f4922a6e6c1ecc8e4bc6b&role.id=8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:25.665240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b001e-e069-4cfd-a8bd-7b9071779855 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.665786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b001e-e069-4cfd-a8bd-7b9071779855 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.665884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b001e-e069-4cfd-a8bd-7b9071779855 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.667233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 656/1288] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:25.672534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.672939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.673111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.690296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:25.691482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:25.720547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:25.758269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 633/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:25.780399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b001e-e069-4cfd-a8bd-7b9071779855 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:25.881472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b001e-e069-4cfd-a8bd-7b9071779855 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:25.891735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:25.915815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:25.916618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:25.916904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:25.917183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:25.917916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:25.919273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:25.931011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:25.932227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:25.933134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 657/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:33:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:25.938106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:26.032946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f0ed6-94f6-4c15-9ace-06e82724293e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:26.066194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:26.066849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:26.067136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:26.067414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:26.068117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:26.068429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:26.069803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7dbf0-0e43-4369-940d-e7af44787685 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:26.074549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 634/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Jun 15 14:33:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:26.081525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:26.179149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840a6e8-502f-43bf-8e68-5469a0793ee3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:26.212249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:26.212914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:26.213202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:26.213502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:26.214288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:26.214605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:26.216083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b68d1-50cc-4326-824d-61e1d16e7443 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:26.221981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 658/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Jun 15 14:33:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:28.327064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc9d13-37a2-44f8-9b2a-d3f8e97d3aec None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.327592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc9d13-37a2-44f8-9b2a-d3f8e97d3aec None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.327684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc9d13-37a2-44f8-9b2a-d3f8e97d3aec None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.328988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 635/1293] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:28.335181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d99673-fe74-4b87-af06-5ee01b34141e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.335604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d99673-fe74-4b87-af06-5ee01b34141e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.335769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d99673-fe74-4b87-af06-5ee01b34141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.352160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-62d99673-fe74-4b87-af06-5ee01b34141e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:28.353337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-62d99673-fe74-4b87-af06-5ee01b34141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:28.382286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:28.419288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 659/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:33:28.442188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc9d13-37a2-44f8-9b2a-d3f8e97d3aec None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:28.473473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:28.543678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc9d13-37a2-44f8-9b2a-d3f8e97d3aec None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:28.576722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a34722-114c-4d23-afcd-b6c5b9df798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:28.577430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a34722-114c-4d23-afcd-b6c5b9df798f None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:28.577722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a34722-114c-4d23-afcd-b6c5b9df798f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:28.578017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a34722-114c-4d23-afcd-b6c5b9df798f None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:28.578760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a34722-114c-4d23-afcd-b6c5b9df798f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:28.580730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a34722-114c-4d23-afcd-b6c5b9df798f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:28.593974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 636/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Jun 15 14:33:28 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:33:30.702926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316f48a-c8d1-450e-b602-0428885f7d05 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:30.704175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316f48a-c8d1-450e-b602-0428885f7d05 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:30.704467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316f48a-c8d1-450e-b602-0428885f7d05 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:30.706511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 660/1296] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:30.712086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:30.712508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:30.712651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:30.729897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:30.731094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:30.802129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 637/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:30.824530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316f48a-c8d1-450e-b602-0428885f7d05 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:30.923584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316f48a-c8d1-450e-b602-0428885f7d05 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:30.956882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:30.957606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:30.957923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:30.958204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:30.958861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:30.960477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:30.973137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:30.974356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:30.975548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 661/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:33:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:30.980320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:30.998084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:31.077626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1fea8-2381-4447-a852-94a38ababd39 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:31.111054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:31.111870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:31.112259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:31.112548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:31.113254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:31.113568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:31.114950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:31.119968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 638/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Jun 15 14:33:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:31.125921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:31.157258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:31.230661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82095848-6d4d-4676-9935-8f192c0b6a9d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:31.265384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:31.266066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:31.266360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:31.266642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:31.267441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] RBAC: Adding query filter params (interface=public, service_id=99934e6c2c304cacbfffb4097588ac5e, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:31.268385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:31.269784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7eb6-a937-42ce-8b24-8cc1f3ea0d5b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:31.274740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 662/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Jun 15 14:33:31 2022] GET /identity/v3/endpoints?service_id=99934e6c2c304cacbfffb4097588ac5e&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:31.280549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:31.360343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:31.378699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55492-6083-42c0-b074-8c6b2b6aa204 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:31.411901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a384c9-2ef3-456c-8d48-3b7f914f49ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:31.412561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a384c9-2ef3-456c-8d48-3b7f914f49ab None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:31.412851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a384c9-2ef3-456c-8d48-3b7f914f49ab None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:31.413133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a384c9-2ef3-456c-8d48-3b7f914f49ab None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:31.413851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a384c9-2ef3-456c-8d48-3b7f914f49ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:31.415212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a384c9-2ef3-456c-8d48-3b7f914f49ab None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:31.421914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 639/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Jun 15 14:33:31 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:33:33.540175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeba9be-2763-49c8-86c6-ff7b4b6f30a3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:33.540728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeba9be-2763-49c8-86c6-ff7b4b6f30a3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:33.540728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeba9be-2763-49c8-86c6-ff7b4b6f30a3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:33.542162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 663/1302] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:33:33.547262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:33.547691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:33.547832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:33.564245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:33:33.566099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:33:33.631735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 640/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:33:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:33:33.655383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeba9be-2763-49c8-86c6-ff7b4b6f30a3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:33.748300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:33.755392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeba9be-2763-49c8-86c6-ff7b4b6f30a3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:33.788100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:33.788895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:33.789183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:33.789465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:33.790152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:33.791516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:33:33.803850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:33:33.805030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Jun 15 14:33:33.806279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 664/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Jun 15 14:33:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:33:33.811050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:33.884980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:33.908095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ff2e-69cc-408c-98bc-e05c441844ea None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:33.941674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:33.942344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:33.942630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:33.942910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:33.943619 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:33:33.943932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:33.945285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952758d2-abc0-4500-9942-f7840e03aee9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:33.950469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 641/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Jun 15 14:33:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:33:33.956795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:33:34.054547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b104d-289f-416e-9eb1-d4e1ba89fea6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:33:34.086809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa45c21-5608-4728-87f8-3c5874e911af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:33:34.087458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa45c21-5608-4728-87f8-3c5874e911af None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:34.087746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa45c21-5608-4728-87f8-3c5874e911af None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:34.088026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa45c21-5608-4728-87f8-3c5874e911af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:34.088759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa45c21-5608-4728-87f8-3c5874e911af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:33:34.090148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa45c21-5608-4728-87f8-3c5874e911af None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:33:34.108589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 665/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Jun 15 14:33:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:33:43.947563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=30 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:33:44.104190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.426203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d484-72f5-4301-861c-cd10afe67e1a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.426785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d484-72f5-4301-861c-cd10afe67e1a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.426785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d484-72f5-4301-861c-cd10afe67e1a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.432553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 642/1307] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:37.433982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b908-bf5c-4409-a7c4-a3faff121045 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.434417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b908-bf5c-4409-a7c4-a3faff121045 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.434553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b908-bf5c-4409-a7c4-a3faff121045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.530014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-16d6b908-bf5c-4409-a7c4-a3faff121045 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:37.531262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-16d6b908-bf5c-4409-a7c4-a3faff121045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:37.649198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 666/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:34:37.705441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdce5-7e12-4c2e-9a93-d49c0aad1330 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.705770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdce5-7e12-4c2e-9a93-d49c0aad1330 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.705975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdce5-7e12-4c2e-9a93-d49c0aad1330 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.707496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 643/1309] 10.222.0.57 () {54 vars in 964 bytes} [Wed Jun 15 14:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:37.714987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62371ff-ab8b-4f9b-9426-89c54dc462e9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.715604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62371ff-ab8b-4f9b-9426-89c54dc462e9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.716126 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62371ff-ab8b-4f9b-9426-89c54dc462e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.888779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c62371ff-ab8b-4f9b-9426-89c54dc462e9 None None] MFA Rules not processed for user `d292189b741b40b8859bfe9883b4c5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:37.891551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c62371ff-ab8b-4f9b-9426-89c54dc462e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:37.931207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.021481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 667/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Wed Jun 15 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:38.030805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdce5-7e12-4c2e-9a93-d49c0aad1330 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.032649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d79fdce5-7e12-4c2e-9a93-d49c0aad1330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:38.153066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdce5-7e12-4c2e-9a93-d49c0aad1330 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.233261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.233863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.234157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.234438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.236086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:38.292730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.353895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.355329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e99a35-3183-4785-a03b-706a117e11c9 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.375902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 644/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:34:40.806698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18c57f-faa4-4a6b-93f1-a34896244ea1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.807279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18c57f-faa4-4a6b-93f1-a34896244ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.807279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18c57f-faa4-4a6b-93f1-a34896244ea1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.808702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 668/1312] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:40.814711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cdc48-147b-436c-a9fe-54ef744ff058 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.815245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cdc48-147b-436c-a9fe-54ef744ff058 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.815356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cdc48-147b-436c-a9fe-54ef744ff058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.906769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4c0cdc48-147b-436c-a9fe-54ef744ff058 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:40.908034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c0cdc48-147b-436c-a9fe-54ef744ff058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:41.012033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 645/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:41.049428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18c57f-faa4-4a6b-93f1-a34896244ea1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.051029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bb18c57f-faa4-4a6b-93f1-a34896244ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:41.155569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18c57f-faa4-4a6b-93f1-a34896244ea1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.228735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.230968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.231665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.232108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.232297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.234584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:41.348772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.350330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42843a9e-db70-4ebe-ad4f-b4472dede382 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.368708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 669/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:43.821722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb179ce0-0ad8-47dc-b04d-d0e2492d0cd7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.822246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb179ce0-0ad8-47dc-b04d-d0e2492d0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.822329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb179ce0-0ad8-47dc-b04d-d0e2492d0cd7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.823661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 646/1315] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:43.828884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd30d2-c60c-4aae-bb21-6ecc56557380 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.829324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd30d2-c60c-4aae-bb21-6ecc56557380 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.829453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd30d2-c60c-4aae-bb21-6ecc56557380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.895198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-64fd30d2-c60c-4aae-bb21-6ecc56557380 None None] MFA Rules not processed for user `892a6e9fd0e24095bfaeb9f4f5859115`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:43.897066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-64fd30d2-c60c-4aae-bb21-6ecc56557380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:43.948703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:43.994605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 670/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:44.034098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb179ce0-0ad8-47dc-b04d-d0e2492d0cd7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.035684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fb179ce0-0ad8-47dc-b04d-d0e2492d0cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:44.045523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.144050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb179ce0-0ad8-47dc-b04d-d0e2492d0cd7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.216259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.216909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.217220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.217501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.219195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:44.281359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.327453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:44.328886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01ef3a-99e6-404d-ae3e-dda741b9ae4b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:44.347659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 647/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:47.011110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfcc9c-de2f-487b-a5c1-7f79498e3761 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.011679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfcc9c-de2f-487b-a5c1-7f79498e3761 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.011794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfcc9c-de2f-487b-a5c1-7f79498e3761 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.013089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 671/1318] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:47.018254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cc6e-b543-49f4-a480-635ec25b6d18 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.018685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cc6e-b543-49f4-a480-635ec25b6d18 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.018821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cc6e-b543-49f4-a480-635ec25b6d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.084582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f4b7cc6e-b543-49f4-a480-635ec25b6d18 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:47.085711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f4b7cc6e-b543-49f4-a480-635ec25b6d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:47.179919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 648/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:47.218609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfcc9c-de2f-487b-a5c1-7f79498e3761 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.220164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-decfcc9c-de2f-487b-a5c1-7f79498e3761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:47.325033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfcc9c-de2f-487b-a5c1-7f79498e3761 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.398287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.398946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.399232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.399510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.401163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:47.409498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.510540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.511993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148d975-2eab-46ee-a673-be01d90ec2bd service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.529855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 672/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:50.338973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58632ca-8104-4dbc-9ac4-fc9f4ecf33fd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.339532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58632ca-8104-4dbc-9ac4-fc9f4ecf33fd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.339532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58632ca-8104-4dbc-9ac4-fc9f4ecf33fd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.340912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 649/1321] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:50.346148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f3319-3db8-4fd7-ad08-4b1b4a0eac04 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.346569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f3319-3db8-4fd7-ad08-4b1b4a0eac04 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.346715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f3319-3db8-4fd7-ad08-4b1b4a0eac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.418511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e01f3319-3db8-4fd7-ad08-4b1b4a0eac04 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:50.419753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e01f3319-3db8-4fd7-ad08-4b1b4a0eac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:50.486313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.529433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 673/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:50.566586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58632ca-8104-4dbc-9ac4-fc9f4ecf33fd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.568360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a58632ca-8104-4dbc-9ac4-fc9f4ecf33fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:50.590649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.675719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58632ca-8104-4dbc-9ac4-fc9f4ecf33fd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.750135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.750796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.751083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.751365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.753026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:50.824550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.865096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.867229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16407353-7894-49c3-bb74-47d42989dc0c service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.885764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 650/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:53.085592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb4d7-a6b1-4b3e-a0a6-a3837f325877 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.086171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb4d7-a6b1-4b3e-a0a6-a3837f325877 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.086171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb4d7-a6b1-4b3e-a0a6-a3837f325877 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.087740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 674/1324] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:53.092939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911e7c-fc0d-4629-b0a8-a543b320bc5c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.093379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911e7c-fc0d-4629-b0a8-a543b320bc5c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.093510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911e7c-fc0d-4629-b0a8-a543b320bc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.159197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-71911e7c-fc0d-4629-b0a8-a543b320bc5c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:53.160406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-71911e7c-fc0d-4629-b0a8-a543b320bc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:53.261263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 651/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:53.312434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a979c-1c1c-4375-9ebc-e321a84910d2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.312731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a979c-1c1c-4375-9ebc-e321a84910d2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.313020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a979c-1c1c-4375-9ebc-e321a84910d2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.314444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 675/1326] 10.222.0.57 () {54 vars in 964 bytes} [Wed Jun 15 14:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:53.321018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b066d-e6c7-484e-b938-09a2cebc5ae5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.321388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b066d-e6c7-484e-b938-09a2cebc5ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.321771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b066d-e6c7-484e-b938-09a2cebc5ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.352910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.402603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-984b066d-e6c7-484e-b938-09a2cebc5ae5 None None] MFA Rules not processed for user `d292189b741b40b8859bfe9883b4c5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:53.403875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-984b066d-e6c7-484e-b938-09a2cebc5ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:53.484728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 652/1327] 10.222.0.57 () {58 vars in 1061 bytes} [Wed Jun 15 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:53.494500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a979c-1c1c-4375-9ebc-e321a84910d2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.496053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-890a979c-1c1c-4375-9ebc-e321a84910d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:53.607417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a979c-1c1c-4375-9ebc-e321a84910d2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.682608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.683267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.683558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.683843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.685504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:53.703233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.799054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:53.800500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661e830-6b3b-45a8-a40b-6cfe094543a9 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.819493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 676/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:56.236004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ddb4-5144-4900-bbe1-94aea86e7f20 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.236555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ddb4-5144-4900-bbe1-94aea86e7f20 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.236555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ddb4-5144-4900-bbe1-94aea86e7f20 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.238034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 653/1329] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:56.244902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ce770-baba-4d6a-855a-2ac9b2e28c8c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.245376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ce770-baba-4d6a-855a-2ac9b2e28c8c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.245376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ce770-baba-4d6a-855a-2ac9b2e28c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.313578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-385ce770-baba-4d6a-855a-2ac9b2e28c8c None None] MFA Rules not processed for user `892a6e9fd0e24095bfaeb9f4f5859115`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:56.314817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-385ce770-baba-4d6a-855a-2ac9b2e28c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:56.380638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:56.414307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 677/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:34:56.451627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100ddb4-5144-4900-bbe1-94aea86e7f20 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.453101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4100ddb4-5144-4900-bbe1-94aea86e7f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:56.558471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100ddb4-5144-4900-bbe1-94aea86e7f20 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.578439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:56.633064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.633767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.634060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.634345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.636008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:56.746603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.748062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e5929-ae3a-4478-a632-807e5e0e8ff2 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.767541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 654/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:34:59.358515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054f404-9013-4d62-9b02-ee43b820318b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.359076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054f404-9013-4d62-9b02-ee43b820318b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.359171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054f404-9013-4d62-9b02-ee43b820318b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.360498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 678/1332] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:59.366987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1752b-092f-47be-8416-4273d57ddcc9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.367485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1752b-092f-47be-8416-4273d57ddcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.367583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1752b-092f-47be-8416-4273d57ddcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.433133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.436321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-75e1752b-092f-47be-8416-4273d57ddcc9 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:59.438156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-75e1752b-092f-47be-8416-4273d57ddcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:59.536798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 655/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:34:59.573205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054f404-9013-4d62-9b02-ee43b820318b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.574772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b054f404-9013-4d62-9b02-ee43b820318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:59.677386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054f404-9013-4d62-9b02-ee43b820318b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.750996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.751648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.751938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.752220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.753911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:59.782069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.863396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.864806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7d13-54e6-41d6-a68f-7699823b6a58 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.883349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 679/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:02.046482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c3947-1e40-4aab-94d3-3d7e3a9bda73 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.047055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c3947-1e40-4aab-94d3-3d7e3a9bda73 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.047055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c3947-1e40-4aab-94d3-3d7e3a9bda73 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.048520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 656/1335] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:02.054282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b33fe-a9f9-4055-aa76-19d8bff9e295 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.054704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b33fe-a9f9-4055-aa76-19d8bff9e295 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.054905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b33fe-a9f9-4055-aa76-19d8bff9e295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.120163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f25b33fe-a9f9-4055-aa76-19d8bff9e295 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:02.121385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f25b33fe-a9f9-4055-aa76-19d8bff9e295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:02.193792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.216217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 680/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:02.256686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c3947-1e40-4aab-94d3-3d7e3a9bda73 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.258296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-232c3947-1e40-4aab-94d3-3d7e3a9bda73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:02.359950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c3947-1e40-4aab-94d3-3d7e3a9bda73 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.387785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.433057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.433730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.434017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.434409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.437051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:02.550499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.551920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36b2d3-966c-414c-9c51-a860e72dce80 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.569403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 657/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:05.143491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c412af-d9d5-4904-bd0b-75e254e343a7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.144034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c412af-d9d5-4904-bd0b-75e254e343a7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.144034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c412af-d9d5-4904-bd0b-75e254e343a7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.145456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 681/1338] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:05.151399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f8176-547e-43b7-9006-d0fb02ed464d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.151809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f8176-547e-43b7-9006-d0fb02ed464d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.151993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f8176-547e-43b7-9006-d0fb02ed464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.217068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3b0f8176-547e-43b7-9006-d0fb02ed464d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:05.218296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3b0f8176-547e-43b7-9006-d0fb02ed464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:05.227015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:05.321484 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 658/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:05.357964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c412af-d9d5-4904-bd0b-75e254e343a7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:05.359493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7c412af-d9d5-4904-bd0b-75e254e343a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:05.466313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c412af-d9d5-4904-bd0b-75e254e343a7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:05.538891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:05.539556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.539849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.540169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.541878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:05.577704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:05.651618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:05.653040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234678f-70e3-41c2-88e5-b9762440b991 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:05.670726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 682/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:09.833013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e9738-7608-49f8-b222-026c515f7dd9 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.833544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e9738-7608-49f8-b222-026c515f7dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.833638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e9738-7608-49f8-b222-026c515f7dd9 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.835302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 659/1341] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:09.840657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c8de8-bcd3-45a9-a311-0475c661c02b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.841074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c8de8-bcd3-45a9-a311-0475c661c02b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.841350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c8de8-bcd3-45a9-a311-0475c661c02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.913889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8c6c8de8-bcd3-45a9-a311-0475c661c02b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:09.915053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8c6c8de8-bcd3-45a9-a311-0475c661c02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:10.010407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.024852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 683/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:10.063322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e9738-7608-49f8-b222-026c515f7dd9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.065495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fe0e9738-7608-49f8-b222-026c515f7dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:10.162331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e9738-7608-49f8-b222-026c515f7dd9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.166480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef62fa-6f04-4b5c-8b1e-6388a499743b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.167129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef62fa-6f04-4b5c-8b1e-6388a499743b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.167412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef62fa-6f04-4b5c-8b1e-6388a499743b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.167689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef62fa-6f04-4b5c-8b1e-6388a499743b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.176724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef62fa-6f04-4b5c-8b1e-6388a499743b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:10.178306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef62fa-6f04-4b5c-8b1e-6388a499743b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:10.187942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 660/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:12.988644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca2d68-0ee9-4d81-89f1-d3c267912133 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.989270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca2d68-0ee9-4d81-89f1-d3c267912133 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.989270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca2d68-0ee9-4d81-89f1-d3c267912133 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.991427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 684/1344] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:12.997646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12558150-98ed-47cf-b994-f2833f1df177 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.998202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12558150-98ed-47cf-b994-f2833f1df177 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.998311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12558150-98ed-47cf-b994-f2833f1df177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:13.029466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-12558150-98ed-47cf-b994-f2833f1df177 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:13.030726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-12558150-98ed-47cf-b994-f2833f1df177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:13.061764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:13.116210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 661/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:13.153406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca2d68-0ee9-4d81-89f1-d3c267912133 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:13.165026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca2d68-0ee9-4d81-89f1-d3c267912133 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:13.168597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8709f-bc7a-4674-aa69-fa80a5f7820f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:13.169254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8709f-bc7a-4674-aa69-fa80a5f7820f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:13.169538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8709f-bc7a-4674-aa69-fa80a5f7820f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:13.169852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8709f-bc7a-4674-aa69-fa80a5f7820f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:13.180606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8709f-bc7a-4674-aa69-fa80a5f7820f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:13.182065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8709f-bc7a-4674-aa69-fa80a5f7820f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:13.191385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 685/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:17.292445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa067c44-0021-4f96-b9ec-3bdc45c35be7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.293002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa067c44-0021-4f96-b9ec-3bdc45c35be7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.293002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa067c44-0021-4f96-b9ec-3bdc45c35be7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.294443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 662/1347] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:17.299552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428b0f-e84a-489a-8c0b-4312a05f26cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.299975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428b0f-e84a-489a-8c0b-4312a05f26cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.300152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428b0f-e84a-489a-8c0b-4312a05f26cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.318099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-17428b0f-e84a-489a-8c0b-4312a05f26cc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:17.320274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-17428b0f-e84a-489a-8c0b-4312a05f26cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:17.328228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 686/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:17.366474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa067c44-0021-4f96-b9ec-3bdc45c35be7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.368395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fa067c44-0021-4f96-b9ec-3bdc45c35be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:17.381744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa067c44-0021-4f96-b9ec-3bdc45c35be7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.385379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504ef2b-5103-4da9-b924-3db3ce25a899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.386069 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ef2b-5103-4da9-b924-3db3ce25a899 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.386360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ef2b-5103-4da9-b924-3db3ce25a899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.386648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ef2b-5103-4da9-b924-3db3ce25a899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.395928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504ef2b-5103-4da9-b924-3db3ce25a899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.397962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504ef2b-5103-4da9-b924-3db3ce25a899 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.407798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 663/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:22.489968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6a72-5302-4ff0-b716-140b605ac20a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.490456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6a72-5302-4ff0-b716-140b605ac20a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.490550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6a72-5302-4ff0-b716-140b605ac20a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.492846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 687/1350] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:22.498173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7f9e0-6a41-4c40-95b0-aee8903a8bd2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.498956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7f9e0-6a41-4c40-95b0-aee8903a8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.499397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7f9e0-6a41-4c40-95b0-aee8903a8bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.577628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5ea7f9e0-6a41-4c40-95b0-aee8903a8bd2 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:22.579487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5ea7f9e0-6a41-4c40-95b0-aee8903a8bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:22.718185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 664/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:35:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:27.058776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b45f-1634-456a-ad8f-1b074a9096d8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:27.059305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b45f-1634-456a-ad8f-1b074a9096d8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:27.059386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b45f-1634-456a-ad8f-1b074a9096d8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:27.061000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 688/1352] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:27.066266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4238-28d3-4455-95c5-d3292de0e8b1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:27.066762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4238-28d3-4455-95c5-d3292de0e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:27.066847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4238-28d3-4455-95c5-d3292de0e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:27.086698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c61d4238-28d3-4455-95c5-d3292de0e8b1 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:27.087922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c61d4238-28d3-4455-95c5-d3292de0e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:27.097071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 665/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:27.312886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=13 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.677181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a01c52-4824-445b-becb-e5494b658f2f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.677876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a01c52-4824-445b-becb-e5494b658f2f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.677876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a01c52-4824-445b-becb-e5494b658f2f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.679844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 689/1354] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:28.685499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac0459-8f4e-4bfb-9ec5-d37b9e8485f8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.685960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac0459-8f4e-4bfb-9ec5-d37b9e8485f8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.686218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac0459-8f4e-4bfb-9ec5-d37b9e8485f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.705239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fdac0459-8f4e-4bfb-9ec5-d37b9e8485f8 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:28.706460 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fdac0459-8f4e-4bfb-9ec5-d37b9e8485f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:28.715071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 666/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:32.471305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f2254-69d1-4de6-9fe9-6283526db6ff None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.471817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f2254-69d1-4de6-9fe9-6283526db6ff None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.471899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f2254-69d1-4de6-9fe9-6283526db6ff None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.473292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 690/1356] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:32.479397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64549-bb1b-46cc-869b-6496a62738e2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.479836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64549-bb1b-46cc-869b-6496a62738e2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.479959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64549-bb1b-46cc-869b-6496a62738e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.498437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f7b64549-bb1b-46cc-869b-6496a62738e2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:32.499611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f7b64549-bb1b-46cc-869b-6496a62738e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:32.507596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 667/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:32.566214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbfcfa-3ba5-4505-88c4-06d0f952e1fb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.566759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbfcfa-3ba5-4505-88c4-06d0f952e1fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.567343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbfcfa-3ba5-4505-88c4-06d0f952e1fb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.568915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 691/1358] 10.222.0.57 () {54 vars in 961 bytes} [Wed Jun 15 14:35:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:32.578773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d584-ce58-4b6a-8221-160d753f8066 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.579755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d584-ce58-4b6a-8221-160d753f8066 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.580245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d584-ce58-4b6a-8221-160d753f8066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.629230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.639840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-40e9d584-ce58-4b6a-8221-160d753f8066 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:32.641029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40e9d584-ce58-4b6a-8221-160d753f8066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:32.748135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 668/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Jun 15 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:32.756135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbfcfa-3ba5-4505-88c4-06d0f952e1fb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.771943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbfcfa-3ba5-4505-88c4-06d0f952e1fb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.778146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a09c1-65d8-4531-a146-d752f0deceef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.778789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a09c1-65d8-4531-a146-d752f0deceef service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.779078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a09c1-65d8-4531-a146-d752f0deceef service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.779376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a09c1-65d8-4531-a146-d752f0deceef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.787812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a09c1-65d8-4531-a146-d752f0deceef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.789453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a09c1-65d8-4531-a146-d752f0deceef service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.799406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 692/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:35.147540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880f73-137e-49f1-bb6f-5b7f38a2a91f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.148661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880f73-137e-49f1-bb6f-5b7f38a2a91f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.148661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880f73-137e-49f1-bb6f-5b7f38a2a91f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.153763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 669/1361] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:35.155384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff04565-4530-4a79-9e21-f9ac40cdf35e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.155783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff04565-4530-4a79-9e21-f9ac40cdf35e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.155948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff04565-4530-4a79-9e21-f9ac40cdf35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.174695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fff04565-4530-4a79-9e21-f9ac40cdf35e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:35.175989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fff04565-4530-4a79-9e21-f9ac40cdf35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:35.184774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 693/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:35.234104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880f73-137e-49f1-bb6f-5b7f38a2a91f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:35.250137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880f73-137e-49f1-bb6f-5b7f38a2a91f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:35.255644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30367b8-202c-4af1-bc49-8f8f07a92a46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:35.256303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30367b8-202c-4af1-bc49-8f8f07a92a46 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.256595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30367b8-202c-4af1-bc49-8f8f07a92a46 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.257439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30367b8-202c-4af1-bc49-8f8f07a92a46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.266593 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30367b8-202c-4af1-bc49-8f8f07a92a46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:35.268220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30367b8-202c-4af1-bc49-8f8f07a92a46 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:35.281728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 670/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:37.806033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5faa1-06bc-4a0a-8cd2-f40fc5d45d05 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.806613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5faa1-06bc-4a0a-8cd2-f40fc5d45d05 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.806613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5faa1-06bc-4a0a-8cd2-f40fc5d45d05 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.808016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 694/1364] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:37.813263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bce08a-16b3-43c7-9348-8a3474e68c5f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.813723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bce08a-16b3-43c7-9348-8a3474e68c5f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.813883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bce08a-16b3-43c7-9348-8a3474e68c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.832494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-44bce08a-16b3-43c7-9348-8a3474e68c5f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:37.834047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-44bce08a-16b3-43c7-9348-8a3474e68c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:37.842707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 671/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:37.888411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85442d-b118-4e71-9208-4fe292a23d4f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.888711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85442d-b118-4e71-9208-4fe292a23d4f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.888995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85442d-b118-4e71-9208-4fe292a23d4f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.890414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 695/1366] 10.222.0.57 () {54 vars in 961 bytes} [Wed Jun 15 14:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:37.895899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833629-4428-4856-b4a4-ef80644407b3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.896225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833629-4428-4856-b4a4-ef80644407b3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.896511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90833629-4428-4856-b4a4-ef80644407b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.914303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-90833629-4428-4856-b4a4-ef80644407b3 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:37.915687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-90833629-4428-4856-b4a4-ef80644407b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:37.931573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 672/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Jun 15 14:35:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:37.939556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85442d-b118-4e71-9208-4fe292a23d4f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.954481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85442d-b118-4e71-9208-4fe292a23d4f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.971579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a1fe2-6ee4-42df-ac9b-d15d616b3ff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.972266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a1fe2-6ee4-42df-ac9b-d15d616b3ff0 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.972729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a1fe2-6ee4-42df-ac9b-d15d616b3ff0 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.974177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a1fe2-6ee4-42df-ac9b-d15d616b3ff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.985720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a1fe2-6ee4-42df-ac9b-d15d616b3ff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.987337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a1fe2-6ee4-42df-ac9b-d15d616b3ff0 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.997319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 696/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:40.241825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36390c1-2a20-4dc1-a2b5-e4b9753f341e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.242433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36390c1-2a20-4dc1-a2b5-e4b9753f341e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.242433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36390c1-2a20-4dc1-a2b5-e4b9753f341e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.243642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 673/1369] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:40.248982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7ed3f-7747-401e-960d-fbb95f5ddc98 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.249417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7ed3f-7747-401e-960d-fbb95f5ddc98 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.249553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7ed3f-7747-401e-960d-fbb95f5ddc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.268459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6ab7ed3f-7747-401e-960d-fbb95f5ddc98 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:40.269759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6ab7ed3f-7747-401e-960d-fbb95f5ddc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:40.278583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 697/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:40.322775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36390c1-2a20-4dc1-a2b5-e4b9753f341e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.338765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36390c1-2a20-4dc1-a2b5-e4b9753f341e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.344618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2de71-27ca-4ff8-9eae-a84a7c64fdfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.345282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2de71-27ca-4ff8-9eae-a84a7c64fdfe service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.345574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2de71-27ca-4ff8-9eae-a84a7c64fdfe service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.345886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2de71-27ca-4ff8-9eae-a84a7c64fdfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.355489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2de71-27ca-4ff8-9eae-a84a7c64fdfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.357076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2de71-27ca-4ff8-9eae-a84a7c64fdfe service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.367892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 674/1371] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:40.695869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bdd75-d74e-496a-a909-1c7b35cc552b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.696163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bdd75-d74e-496a-a909-1c7b35cc552b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.696625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bdd75-d74e-496a-a909-1c7b35cc552b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.698049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 698/1372] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:40.703507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de978df-f83a-4eb9-9608-ed03c9465935 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.704000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de978df-f83a-4eb9-9608-ed03c9465935 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.704292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de978df-f83a-4eb9-9608-ed03c9465935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.722646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6de978df-f83a-4eb9-9608-ed03c9465935 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:40.723836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6de978df-f83a-4eb9-9608-ed03c9465935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:40.731870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 675/1373] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:41.039524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2557-e8e1-4d5e-90e9-533714a22921 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.039892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2557-e8e1-4d5e-90e9-533714a22921 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.040246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2557-e8e1-4d5e-90e9-533714a22921 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.041749 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 699/1374] 10.222.0.57 () {54 vars in 963 bytes} [Wed Jun 15 14:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:41.047494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2132166-c42a-4d1b-a1b3-fb12c9b7b696 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.047791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2132166-c42a-4d1b-a1b3-fb12c9b7b696 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.048076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2132166-c42a-4d1b-a1b3-fb12c9b7b696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.101216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b2132166-c42a-4d1b-a1b3-fb12c9b7b696 None None] MFA Rules not processed for user `a26d10743fe844a98ecfcae3974dbce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:41.102442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b2132166-c42a-4d1b-a1b3-fb12c9b7b696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:41.166121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 676/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Wed Jun 15 14:35:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:41.173710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2557-e8e1-4d5e-90e9-533714a22921 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.185153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2557-e8e1-4d5e-90e9-533714a22921 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.189088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5eb0b-96b6-4b6f-bb43-e256320d9bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.189852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5eb0b-96b6-4b6f-bb43-e256320d9bed service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.190374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5eb0b-96b6-4b6f-bb43-e256320d9bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.190748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5eb0b-96b6-4b6f-bb43-e256320d9bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.201525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5eb0b-96b6-4b6f-bb43-e256320d9bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.203074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5eb0b-96b6-4b6f-bb43-e256320d9bed service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.213926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 700/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:42.560236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853f91-9c6f-428c-88b2-a689cc54195f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.560776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853f91-9c6f-428c-88b2-a689cc54195f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.560776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853f91-9c6f-428c-88b2-a689cc54195f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.562223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 677/1377] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:42.567653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da101b18-fd6c-43fd-8d82-d226783b4acb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.568080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da101b18-fd6c-43fd-8d82-d226783b4acb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.568255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da101b18-fd6c-43fd-8d82-d226783b4acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.587836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-da101b18-fd6c-43fd-8d82-d226783b4acb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:42.589050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da101b18-fd6c-43fd-8d82-d226783b4acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:42.597155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 701/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:42.641420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5853f91-9c6f-428c-88b2-a689cc54195f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:42.656461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5853f91-9c6f-428c-88b2-a689cc54195f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:42.662028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ed20a-21eb-4a93-9d3b-ed9f2e0d3474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.662682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ed20a-21eb-4a93-9d3b-ed9f2e0d3474 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.662971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ed20a-21eb-4a93-9d3b-ed9f2e0d3474 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.663272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ed20a-21eb-4a93-9d3b-ed9f2e0d3474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.671845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ed20a-21eb-4a93-9d3b-ed9f2e0d3474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.673445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ed20a-21eb-4a93-9d3b-ed9f2e0d3474 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.677607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:42.684732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 678/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:44.921784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90460d-397c-4bcf-b456-752a6a8c3223 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.922363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90460d-397c-4bcf-b456-752a6a8c3223 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.922363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90460d-397c-4bcf-b456-752a6a8c3223 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.923717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 702/1380] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:44.929722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017e522-6702-4957-9a98-247168a1327e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.930550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017e522-6702-4957-9a98-247168a1327e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.930855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017e522-6702-4957-9a98-247168a1327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.948646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7017e522-6702-4957-9a98-247168a1327e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:44.950142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7017e522-6702-4957-9a98-247168a1327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:44.959119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 679/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:45.004883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90460d-397c-4bcf-b456-752a6a8c3223 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:45.019800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90460d-397c-4bcf-b456-752a6a8c3223 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.025623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333130-440c-48b0-abfb-deef41aba737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.026452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333130-440c-48b0-abfb-deef41aba737 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.026754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333130-440c-48b0-abfb-deef41aba737 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.027038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333130-440c-48b0-abfb-deef41aba737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.035547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333130-440c-48b0-abfb-deef41aba737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:45.037158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333130-440c-48b0-abfb-deef41aba737 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:45.047842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 703/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:47.245205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216756-a7b1-4833-a0ab-a3e16f424911 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.245977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216756-a7b1-4833-a0ab-a3e16f424911 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.245977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216756-a7b1-4833-a0ab-a3e16f424911 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.250018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 680/1383] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:47.252876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5bd5-1825-44d9-8686-30dd74ec2f3b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.253320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5bd5-1825-44d9-8686-30dd74ec2f3b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.253434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5bd5-1825-44d9-8686-30dd74ec2f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.272398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4c8a5bd5-1825-44d9-8686-30dd74ec2f3b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:47.273891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4c8a5bd5-1825-44d9-8686-30dd74ec2f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:47.285238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 704/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:47.324764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216756-a7b1-4833-a0ab-a3e16f424911 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.339857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216756-a7b1-4833-a0ab-a3e16f424911 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.345727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b736166-ab49-4165-b4ec-41695a1f8afe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.346417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736166-ab49-4165-b4ec-41695a1f8afe service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.346804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736166-ab49-4165-b4ec-41695a1f8afe service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.347184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736166-ab49-4165-b4ec-41695a1f8afe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.356951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b736166-ab49-4165-b4ec-41695a1f8afe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.358667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b736166-ab49-4165-b4ec-41695a1f8afe service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.368746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 681/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:49.512870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b897d5-dfc4-4d2a-a88d-f952563d0a07 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.513410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b897d5-dfc4-4d2a-a88d-f952563d0a07 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.513410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b897d5-dfc4-4d2a-a88d-f952563d0a07 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.514773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 705/1386] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:49.520173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2882-8535-4866-9de1-8aa37ba0a55f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.520719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2882-8535-4866-9de1-8aa37ba0a55f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.520719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2882-8535-4866-9de1-8aa37ba0a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.544000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4a6a2882-8535-4866-9de1-8aa37ba0a55f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:49.545981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4a6a2882-8535-4866-9de1-8aa37ba0a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:49.557512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 682/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:49.603781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b897d5-dfc4-4d2a-a88d-f952563d0a07 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.618756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b897d5-dfc4-4d2a-a88d-f952563d0a07 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.624191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd9fa4-1bdb-4ab7-b2ce-6a7040c0b148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.624837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd9fa4-1bdb-4ab7-b2ce-6a7040c0b148 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.625127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd9fa4-1bdb-4ab7-b2ce-6a7040c0b148 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.625408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd9fa4-1bdb-4ab7-b2ce-6a7040c0b148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.635100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd9fa4-1bdb-4ab7-b2ce-6a7040c0b148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.636674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd9fa4-1bdb-4ab7-b2ce-6a7040c0b148 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.646520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 706/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:35:51.820113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b07ef-a63f-4737-95f4-25dc169d34f9 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.820938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b07ef-a63f-4737-95f4-25dc169d34f9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.820938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b07ef-a63f-4737-95f4-25dc169d34f9 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.822305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 683/1389] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:51.828652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42fcf6-709f-48d5-a4e6-634ae7440515 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.829120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42fcf6-709f-48d5-a4e6-634ae7440515 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.829378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42fcf6-709f-48d5-a4e6-634ae7440515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.853431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-db42fcf6-709f-48d5-a4e6-634ae7440515 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:51.855108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db42fcf6-709f-48d5-a4e6-634ae7440515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:51.864814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 707/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:51.914177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b07ef-a63f-4737-95f4-25dc169d34f9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.930536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b07ef-a63f-4737-95f4-25dc169d34f9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:51.936414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a365d3-79b0-4b40-bc84-29dafac5b351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:51.937119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a365d3-79b0-4b40-bc84-29dafac5b351 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.937447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a365d3-79b0-4b40-bc84-29dafac5b351 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.937816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a365d3-79b0-4b40-bc84-29dafac5b351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.947541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a365d3-79b0-4b40-bc84-29dafac5b351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.949327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a365d3-79b0-4b40-bc84-29dafac5b351 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.960766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 684/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:54.124617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e64ed1-4c63-4a4a-befc-5b64c6bd1d92 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.125172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e64ed1-4c63-4a4a-befc-5b64c6bd1d92 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.125172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e64ed1-4c63-4a4a-befc-5b64c6bd1d92 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.126749 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 708/1392] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:54.132074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23e87c-4e31-4731-a143-d12e301bef27 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.132511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23e87c-4e31-4731-a143-d12e301bef27 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.132652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23e87c-4e31-4731-a143-d12e301bef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.150929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7a23e87c-4e31-4731-a143-d12e301bef27 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:54.152140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7a23e87c-4e31-4731-a143-d12e301bef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:54.160151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 685/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:54.204463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e64ed1-4c63-4a4a-befc-5b64c6bd1d92 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.219653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e64ed1-4c63-4a4a-befc-5b64c6bd1d92 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.226018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901f622-2440-4753-93a1-d796f9e67ab2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.226674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901f622-2440-4753-93a1-d796f9e67ab2 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.226969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901f622-2440-4753-93a1-d796f9e67ab2 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.227256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901f622-2440-4753-93a1-d796f9e67ab2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.235976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901f622-2440-4753-93a1-d796f9e67ab2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.237786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901f622-2440-4753-93a1-d796f9e67ab2 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.247821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 709/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:56.452591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f54ae6-3209-468d-a4c4-d834a9ea44da None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.453239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f54ae6-3209-468d-a4c4-d834a9ea44da None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.453239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f54ae6-3209-468d-a4c4-d834a9ea44da None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.454612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 686/1395] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:56.460567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f574a0f-0b22-45d9-a865-117980cdb716 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.461022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f574a0f-0b22-45d9-a865-117980cdb716 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.461173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f574a0f-0b22-45d9-a865-117980cdb716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.479700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9f574a0f-0b22-45d9-a865-117980cdb716 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:56.480888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9f574a0f-0b22-45d9-a865-117980cdb716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:56.488728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 710/1396] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:56.531613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f54ae6-3209-468d-a4c4-d834a9ea44da None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.546698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f54ae6-3209-468d-a4c4-d834a9ea44da None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.551995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7564a-f237-4386-b6a9-9360ba005359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.552900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7564a-f237-4386-b6a9-9360ba005359 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.553266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7564a-f237-4386-b6a9-9360ba005359 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.553551 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7564a-f237-4386-b6a9-9360ba005359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.561902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7564a-f237-4386-b6a9-9360ba005359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.563702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7564a-f237-4386-b6a9-9360ba005359 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.573206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 687/1397] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:35:56.994470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35385d-c070-4121-8ff3-dba41fe6b0c3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.994769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35385d-c070-4121-8ff3-dba41fe6b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.995050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35385d-c070-4121-8ff3-dba41fe6b0c3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.996581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 711/1398] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:57.001729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb9802-82e3-498e-844f-361675be0d12 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.002134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb9802-82e3-498e-844f-361675be0d12 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.002487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb9802-82e3-498e-844f-361675be0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.021648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-71cb9802-82e3-498e-844f-361675be0d12 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:57.022879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-71cb9802-82e3-498e-844f-361675be0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:57.030858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 688/1399] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:57.088505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60144-c4ff-4bf4-9c12-999b7b0d630c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.088800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60144-c4ff-4bf4-9c12-999b7b0d630c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.089083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60144-c4ff-4bf4-9c12-999b7b0d630c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.090758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 712/1400] 10.222.0.57 () {54 vars in 962 bytes} [Wed Jun 15 14:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:57.096087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b65ea-4c52-4c5d-8866-2b5ab8222bab None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.096476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b65ea-4c52-4c5d-8866-2b5ab8222bab None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.096832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b65ea-4c52-4c5d-8866-2b5ab8222bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.147664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-509b65ea-4c52-4c5d-8866-2b5ab8222bab None None] MFA Rules not processed for user `3ca2cd515aa84a3ca01656977036ac07`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:57.148862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-509b65ea-4c52-4c5d-8866-2b5ab8222bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:57.208747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 689/1401] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Jun 15 14:35:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:57.216477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60144-c4ff-4bf4-9c12-999b7b0d630c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.227801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60144-c4ff-4bf4-9c12-999b7b0d630c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.231314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0344e-18e0-4e2a-9092-521b7d6dda21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.231961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0344e-18e0-4e2a-9092-521b7d6dda21 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.232246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0344e-18e0-4e2a-9092-521b7d6dda21 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.232525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0344e-18e0-4e2a-9092-521b7d6dda21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.242591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0344e-18e0-4e2a-9092-521b7d6dda21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.244027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0344e-18e0-4e2a-9092-521b7d6dda21 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.253575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 713/1402] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:35:57.974655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ac3fa-8875-4a2a-aea2-8eee4393a817 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.975201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ac3fa-8875-4a2a-aea2-8eee4393a817 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.975316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ac3fa-8875-4a2a-aea2-8eee4393a817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.023524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c77ac3fa-8875-4a2a-aea2-8eee4393a817 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:58.024738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c77ac3fa-8875-4a2a-aea2-8eee4393a817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:58.064062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:58.088389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 690/1403] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:35:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:58.134847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21232553-3eb9-4828-98ed-ab6c68d664a7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.135344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21232553-3eb9-4828-98ed-ab6c68d664a7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.135612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21232553-3eb9-4828-98ed-ab6c68d664a7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.137171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 714/1404] 10.222.0.57 () {54 vars in 942 bytes} [Wed Jun 15 14:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:58.143575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2c2ea-f4e4-4be5-a0b5-d5d73a42149c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.143877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2c2ea-f4e4-4be5-a0b5-d5d73a42149c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.144168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2c2ea-f4e4-4be5-a0b5-d5d73a42149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.194496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-95b2c2ea-f4e4-4be5-a0b5-d5d73a42149c None None] MFA Rules not processed for user `cdf6e3c71de545d782c99ca87d86fb1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:58.195703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-95b2c2ea-f4e4-4be5-a0b5-d5d73a42149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:58.269592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 691/1405] 10.222.0.57 () {58 vars in 1039 bytes} [Wed Jun 15 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:35:58.279201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21232553-3eb9-4828-98ed-ab6c68d664a7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:58.287253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=61 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:58.295183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21232553-3eb9-4828-98ed-ab6c68d664a7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:58.302014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0832546-66ee-4552-8c68-6a92653dd712 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:58.302876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0832546-66ee-4552-8c68-6a92653dd712 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.303840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0832546-66ee-4552-8c68-6a92653dd712 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.304252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0832546-66ee-4552-8c68-6a92653dd712 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.313785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0832546-66ee-4552-8c68-6a92653dd712 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:58.315378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0832546-66ee-4552-8c68-6a92653dd712 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:58.323955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 715/1406] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:35:58.778450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8a22f-eee6-4efc-b337-6ba05cdaf500 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.778816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8a22f-eee6-4efc-b337-6ba05cdaf500 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.779030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8a22f-eee6-4efc-b337-6ba05cdaf500 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.780550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 692/1407] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:58.785696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00639463-049c-406a-90f7-4837e4fe5cf7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.786012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00639463-049c-406a-90f7-4837e4fe5cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.786409 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00639463-049c-406a-90f7-4837e4fe5cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.807232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-00639463-049c-406a-90f7-4837e4fe5cf7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:58.808424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-00639463-049c-406a-90f7-4837e4fe5cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:58.816612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 716/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:35:58.862128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8a22f-eee6-4efc-b337-6ba05cdaf500 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:58.877262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8a22f-eee6-4efc-b337-6ba05cdaf500 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:58.883452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4029d30-89f6-426f-b758-3e0893308fbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:58.884103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4029d30-89f6-426f-b758-3e0893308fbe service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.884395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4029d30-89f6-426f-b758-3e0893308fbe service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.884680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4029d30-89f6-426f-b758-3e0893308fbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.894260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4029d30-89f6-426f-b758-3e0893308fbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:58.895952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4029d30-89f6-426f-b758-3e0893308fbe service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:58.906161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 693/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:08.801909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:08.901557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=59 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.125134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6491a-458f-4a03-bdb4-845810bebf5d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.125771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6491a-458f-4a03-bdb4-845810bebf5d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.125771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6491a-458f-4a03-bdb4-845810bebf5d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.127189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 717/1410] 10.222.0.57 () {54 vars in 918 bytes} [Wed Jun 15 14:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:21.133892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff9610-5a93-4b52-8272-91a3ac135816 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.134399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff9610-5a93-4b52-8272-91a3ac135816 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.134534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff9610-5a93-4b52-8272-91a3ac135816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.153216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-27ff9610-5a93-4b52-8272-91a3ac135816 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:21.154738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-27ff9610-5a93-4b52-8272-91a3ac135816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:21.163669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 694/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Jun 15 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:21.179185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5962-a8d9-4deb-bae0-5efdb6ffc1f4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.179501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5962-a8d9-4deb-bae0-5efdb6ffc1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.179787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5962-a8d9-4deb-bae0-5efdb6ffc1f4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.181161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 718/1412] 10.222.0.57 () {54 vars in 918 bytes} [Wed Jun 15 14:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:21.186968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31b4a1-87e2-4a15-a844-88634eb175d4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.187269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31b4a1-87e2-4a15-a844-88634eb175d4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.187561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31b4a1-87e2-4a15-a844-88634eb175d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.212939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-df31b4a1-87e2-4a15-a844-88634eb175d4 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:21.214438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-df31b4a1-87e2-4a15-a844-88634eb175d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:21.224554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 695/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Jun 15 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:23.848606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22a00-2ff7-4192-a213-722e76b87637 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:23.849181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22a00-2ff7-4192-a213-722e76b87637 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:23.849181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22a00-2ff7-4192-a213-722e76b87637 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:23.850610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:23.856191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f182f-bbf0-40c2-8750-1a5ee96241c2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:23.856627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f182f-bbf0-40c2-8750-1a5ee96241c2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:23.856758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f182f-bbf0-40c2-8750-1a5ee96241c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:23.874734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f87f182f-bbf0-40c2-8750-1a5ee96241c2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:23.875924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f87f182f-bbf0-40c2-8750-1a5ee96241c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:23.884079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:23.928457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce22a00-2ff7-4192-a213-722e76b87637 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:23.943227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce22a00-2ff7-4192-a213-722e76b87637 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:23.948682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be144917-eb2b-4732-b09a-591eb41af578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:23.949355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be144917-eb2b-4732-b09a-591eb41af578 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:23.949704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be144917-eb2b-4732-b09a-591eb41af578 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:23.949978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be144917-eb2b-4732-b09a-591eb41af578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:23.958025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be144917-eb2b-4732-b09a-591eb41af578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:23.959599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be144917-eb2b-4732-b09a-591eb41af578 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:23.969157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 720/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:24.916830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583f64e-dd9f-49c3-86a3-d95f8d3511b3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.917410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583f64e-dd9f-49c3-86a3-d95f8d3511b3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.917410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583f64e-dd9f-49c3-86a3-d95f8d3511b3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.918814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:24.923736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08726e8e-9295-426c-829f-67022b2d8e7f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.924170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08726e8e-9295-426c-829f-67022b2d8e7f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.924402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08726e8e-9295-426c-829f-67022b2d8e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.942424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-08726e8e-9295-426c-829f-67022b2d8e7f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:24.943634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-08726e8e-9295-426c-829f-67022b2d8e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:24.951549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:25.118808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583f64e-dd9f-49c3-86a3-d95f8d3511b3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:25.133385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583f64e-dd9f-49c3-86a3-d95f8d3511b3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:25.138823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752e8a2-cabd-4ca2-a43a-795717954102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:25.139464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752e8a2-cabd-4ca2-a43a-795717954102 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:25.139756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752e8a2-cabd-4ca2-a43a-795717954102 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:25.140041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752e8a2-cabd-4ca2-a43a-795717954102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:25.148776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752e8a2-cabd-4ca2-a43a-795717954102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:25.150410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752e8a2-cabd-4ca2-a43a-795717954102 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:25.160515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 698/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:26.095227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dd9bf-47ce-473d-a376-07f1ad9d4fba None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.095894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dd9bf-47ce-473d-a376-07f1ad9d4fba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.095894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dd9bf-47ce-473d-a376-07f1ad9d4fba None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.097952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:26.103184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ebe44-1d93-43b3-a921-aa1cf72b6227 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.104154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ebe44-1d93-43b3-a921-aa1cf72b6227 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.104154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ebe44-1d93-43b3-a921-aa1cf72b6227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.123066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ba4ebe44-1d93-43b3-a921-aa1cf72b6227 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:26.124275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ba4ebe44-1d93-43b3-a921-aa1cf72b6227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:26.132070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:26.284093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dd9bf-47ce-473d-a376-07f1ad9d4fba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:26.298515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dd9bf-47ce-473d-a376-07f1ad9d4fba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:26.304663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855098d5-132b-4186-a493-f5b0f318a88b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:26.305307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855098d5-132b-4186-a493-f5b0f318a88b service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.305597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855098d5-132b-4186-a493-f5b0f318a88b service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.305914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855098d5-132b-4186-a493-f5b0f318a88b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.313980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855098d5-132b-4186-a493-f5b0f318a88b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:26.315996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855098d5-132b-4186-a493-f5b0f318a88b service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:26.325436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 723/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:27.240883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9758b31-21ec-49b3-93fe-b5f2d0b6da6a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.241537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9758b31-21ec-49b3-93fe-b5f2d0b6da6a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.241537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9758b31-21ec-49b3-93fe-b5f2d0b6da6a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.242822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:27.247863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998b1ca-e29e-4731-a867-b14a330d3fc3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.248323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998b1ca-e29e-4731-a867-b14a330d3fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.248429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998b1ca-e29e-4731-a867-b14a330d3fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.267178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7998b1ca-e29e-4731-a867-b14a330d3fc3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:27.268795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7998b1ca-e29e-4731-a867-b14a330d3fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:27.276645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:27.428981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9758b31-21ec-49b3-93fe-b5f2d0b6da6a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.444583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9758b31-21ec-49b3-93fe-b5f2d0b6da6a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.449992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d03e0b-28a6-47fe-9e56-927a9317d924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.450649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d03e0b-28a6-47fe-9e56-927a9317d924 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.450936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d03e0b-28a6-47fe-9e56-927a9317d924 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.451215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d03e0b-28a6-47fe-9e56-927a9317d924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.459842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d03e0b-28a6-47fe-9e56-927a9317d924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.461409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d03e0b-28a6-47fe-9e56-927a9317d924 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.470920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 701/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:28.376859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629c5b-2a91-4e1c-a7e9-0c3d83a3d830 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.378000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629c5b-2a91-4e1c-a7e9-0c3d83a3d830 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.378497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629c5b-2a91-4e1c-a7e9-0c3d83a3d830 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.379881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:28.385042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9884-727d-4177-8c2a-73d2a20fb19a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.385492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9884-727d-4177-8c2a-73d2a20fb19a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.385632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9884-727d-4177-8c2a-73d2a20fb19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.404152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5f8f9884-727d-4177-8c2a-73d2a20fb19a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:28.405344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5f8f9884-727d-4177-8c2a-73d2a20fb19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:28.413372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:28.566770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22629c5b-2a91-4e1c-a7e9-0c3d83a3d830 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.581403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22629c5b-2a91-4e1c-a7e9-0c3d83a3d830 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.586704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bc877-248a-4597-a59c-652e1cdcf271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.587339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bc877-248a-4597-a59c-652e1cdcf271 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.587625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bc877-248a-4597-a59c-652e1cdcf271 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.587976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bc877-248a-4597-a59c-652e1cdcf271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.596801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bc877-248a-4597-a59c-652e1cdcf271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:28.598440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bc877-248a-4597-a59c-652e1cdcf271 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:28.607675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 726/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:29.461854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33bf4-3263-4a37-a44d-343659315c94 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.462287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33bf4-3263-4a37-a44d-343659315c94 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.462427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33bf4-3263-4a37-a44d-343659315c94 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.464067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:29.468916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9a8d9-555f-4e9b-9e44-cf67ca32f099 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.469312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9a8d9-555f-4e9b-9e44-cf67ca32f099 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.469488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9a8d9-555f-4e9b-9e44-cf67ca32f099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.488534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ffd9a8d9-555f-4e9b-9e44-cf67ca32f099 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:29.489769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffd9a8d9-555f-4e9b-9e44-cf67ca32f099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:29.497518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:29.651037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b33bf4-3263-4a37-a44d-343659315c94 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:29.666118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b33bf4-3263-4a37-a44d-343659315c94 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.671489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e788053-5860-4084-b687-9cf7f19373d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:29.672130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e788053-5860-4084-b687-9cf7f19373d0 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.672416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e788053-5860-4084-b687-9cf7f19373d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.672701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e788053-5860-4084-b687-9cf7f19373d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.680756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e788053-5860-4084-b687-9cf7f19373d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:29.682382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e788053-5860-4084-b687-9cf7f19373d0 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:29.691709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 704/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:30.547351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143accb-0ffc-4a5c-a5c7-abb9bd7e4c97 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.547985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143accb-0ffc-4a5c-a5c7-abb9bd7e4c97 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.547985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143accb-0ffc-4a5c-a5c7-abb9bd7e4c97 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.549284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:30.554385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ac538-f417-4ef5-828f-7af069089d18 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.554780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ac538-f417-4ef5-828f-7af069089d18 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.554962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ac538-f417-4ef5-828f-7af069089d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.573114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f51ac538-f417-4ef5-828f-7af069089d18 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:30.574354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f51ac538-f417-4ef5-828f-7af069089d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:30.582715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:30.732478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143accb-0ffc-4a5c-a5c7-abb9bd7e4c97 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.746740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143accb-0ffc-4a5c-a5c7-abb9bd7e4c97 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.752660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0315be-e558-4d52-a9e4-dfd0212672a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.753302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0315be-e558-4d52-a9e4-dfd0212672a3 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.753590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0315be-e558-4d52-a9e4-dfd0212672a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.753902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0315be-e558-4d52-a9e4-dfd0212672a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.761968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0315be-e558-4d52-a9e4-dfd0212672a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.763538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0315be-e558-4d52-a9e4-dfd0212672a3 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.773864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 729/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:31.633386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350cbfa-5e1d-4315-9164-b114e937a42f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.633947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350cbfa-5e1d-4315-9164-b114e937a42f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.633947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350cbfa-5e1d-4315-9164-b114e937a42f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.635315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:31.640336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1171d-4dc0-4a52-8487-a65b2d18e3b6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.640779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1171d-4dc0-4a52-8487-a65b2d18e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.640905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1171d-4dc0-4a52-8487-a65b2d18e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.660416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-72d1171d-4dc0-4a52-8487-a65b2d18e3b6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:31.661601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-72d1171d-4dc0-4a52-8487-a65b2d18e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:31.669356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:31.824746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350cbfa-5e1d-4315-9164-b114e937a42f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:31.839629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350cbfa-5e1d-4315-9164-b114e937a42f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:31.844964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7f9f1-fb16-4f0f-9379-d1af75ca7856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:31.846571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7f9f1-fb16-4f0f-9379-d1af75ca7856 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.846571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7f9f1-fb16-4f0f-9379-d1af75ca7856 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.847358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7f9f1-fb16-4f0f-9379-d1af75ca7856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.854712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7f9f1-fb16-4f0f-9379-d1af75ca7856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:31.856264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7f9f1-fb16-4f0f-9379-d1af75ca7856 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:31.865445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 707/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:32.730470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221c0e0-d343-4771-adf7-71ca5f8b4dde None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.731029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221c0e0-d343-4771-adf7-71ca5f8b4dde None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.731029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221c0e0-d343-4771-adf7-71ca5f8b4dde None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.732319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:32.737867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb872c-63e7-4e38-a5a7-6168a878432d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.738283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb872c-63e7-4e38-a5a7-6168a878432d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.738433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb872c-63e7-4e38-a5a7-6168a878432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.756699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b6fb872c-63e7-4e38-a5a7-6168a878432d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:32.757952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b6fb872c-63e7-4e38-a5a7-6168a878432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:32.765804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:32.917577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221c0e0-d343-4771-adf7-71ca5f8b4dde None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:32.932684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221c0e0-d343-4771-adf7-71ca5f8b4dde None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:32.938661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a49975-29db-43f9-a0d3-e53f709e6394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:32.939864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49975-29db-43f9-a0d3-e53f709e6394 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.940154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49975-29db-43f9-a0d3-e53f709e6394 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.940432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49975-29db-43f9-a0d3-e53f709e6394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.948575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a49975-29db-43f9-a0d3-e53f709e6394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:32.950161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a49975-29db-43f9-a0d3-e53f709e6394 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:32.959530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 732/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:35.027609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c995-a0c3-4504-99cf-5684fb985a8d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:35.028154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c995-a0c3-4504-99cf-5684fb985a8d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:35.028154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c995-a0c3-4504-99cf-5684fb985a8d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:35.029626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:35.035726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e4473-034a-4a3d-8246-e8a4783cfd2c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:35.036171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e4473-034a-4a3d-8246-e8a4783cfd2c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:35.036279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e4473-034a-4a3d-8246-e8a4783cfd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:35.054224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f43e4473-034a-4a3d-8246-e8a4783cfd2c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:35.055425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f43e4473-034a-4a3d-8246-e8a4783cfd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:35.063224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:35.102492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c995-a0c3-4504-99cf-5684fb985a8d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:35.114181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c995-a0c3-4504-99cf-5684fb985a8d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:35.117604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15127d-dca9-419a-895a-a8949cb3aff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:35.118267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15127d-dca9-419a-895a-a8949cb3aff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:35.118681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15127d-dca9-419a-895a-a8949cb3aff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:35.118920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15127d-dca9-419a-895a-a8949cb3aff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:35.128342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15127d-dca9-419a-895a-a8949cb3aff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:35.129785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15127d-dca9-419a-895a-a8949cb3aff7 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:35.139165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 710/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:37.409676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3994cc1-c168-40ac-8d5b-18c39c7ed5f0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.410239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3994cc1-c168-40ac-8d5b-18c39c7ed5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.410239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3994cc1-c168-40ac-8d5b-18c39c7ed5f0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.411641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:37.417128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d117b-f5b4-4115-a31e-3545fea966ad None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.417573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d117b-f5b4-4115-a31e-3545fea966ad None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.417753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d117b-f5b4-4115-a31e-3545fea966ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.435483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4c1d117b-f5b4-4115-a31e-3545fea966ad None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:37.436683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c1d117b-f5b4-4115-a31e-3545fea966ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:37.445110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:37.483438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3994cc1-c168-40ac-8d5b-18c39c7ed5f0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.494964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3994cc1-c168-40ac-8d5b-18c39c7ed5f0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.498524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebdc55-f9de-4b85-8e9d-7770620e2aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.499168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebdc55-f9de-4b85-8e9d-7770620e2aae service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.499459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebdc55-f9de-4b85-8e9d-7770620e2aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.499743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebdc55-f9de-4b85-8e9d-7770620e2aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.509164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebdc55-f9de-4b85-8e9d-7770620e2aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:37.510631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebdc55-f9de-4b85-8e9d-7770620e2aae service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:37.520610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 735/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:39.988524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c2b-756c-4065-abf8-e4e3035d8545 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.989033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c2b-756c-4065-abf8-e4e3035d8545 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.989033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c2b-756c-4065-abf8-e4e3035d8545 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:39.990427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:39.995936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194d84-44df-4757-9931-e25d2c62e819 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.996339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194d84-44df-4757-9931-e25d2c62e819 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.996504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194d84-44df-4757-9931-e25d2c62e819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:40.014405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c2194d84-44df-4757-9931-e25d2c62e819 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:40.015632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c2194d84-44df-4757-9931-e25d2c62e819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:40.023470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:40.062189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c2b-756c-4065-abf8-e4e3035d8545 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:40.073463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c2b-756c-4065-abf8-e4e3035d8545 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:40.077024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ee9fb-a4cf-411c-b808-c8c93128fe11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:40.077682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee9fb-a4cf-411c-b808-c8c93128fe11 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:40.077975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee9fb-a4cf-411c-b808-c8c93128fe11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:40.078257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee9fb-a4cf-411c-b808-c8c93128fe11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:40.087274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ee9fb-a4cf-411c-b808-c8c93128fe11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:40.089237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ee9fb-a4cf-411c-b808-c8c93128fe11 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:40.098647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 713/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:42.471588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da6bb8-5eb2-40fb-8d80-0d3350b00912 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:42.472077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da6bb8-5eb2-40fb-8d80-0d3350b00912 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:42.472183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da6bb8-5eb2-40fb-8d80-0d3350b00912 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:42.473535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:42.478726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a67e3-4d20-4a52-93be-81e8ee19bc06 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:42.479201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a67e3-4d20-4a52-93be-81e8ee19bc06 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:42.479324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a67e3-4d20-4a52-93be-81e8ee19bc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:42.496951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6f6a67e3-4d20-4a52-93be-81e8ee19bc06 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:42.498189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f6a67e3-4d20-4a52-93be-81e8ee19bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:42.506530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:42.545369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da6bb8-5eb2-40fb-8d80-0d3350b00912 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:42.556328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da6bb8-5eb2-40fb-8d80-0d3350b00912 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:42.559795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60723d74-9892-496a-ac0a-92218db15459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:42.560425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60723d74-9892-496a-ac0a-92218db15459 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:42.560710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60723d74-9892-496a-ac0a-92218db15459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:42.560984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60723d74-9892-496a-ac0a-92218db15459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:42.570716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60723d74-9892-496a-ac0a-92218db15459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:42.572140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60723d74-9892-496a-ac0a-92218db15459 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:42.582423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 738/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:44.784678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977733fa-5ad9-467a-bd24-80ffee4d8ac3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:44.785219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977733fa-5ad9-467a-bd24-80ffee4d8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:44.785219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977733fa-5ad9-467a-bd24-80ffee4d8ac3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:44.786733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:44.792026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010483e3-e27b-4440-b6fb-5bc51877cd95 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:44.792479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010483e3-e27b-4440-b6fb-5bc51877cd95 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:44.792609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010483e3-e27b-4440-b6fb-5bc51877cd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:44.810795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-010483e3-e27b-4440-b6fb-5bc51877cd95 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:44.811996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-010483e3-e27b-4440-b6fb-5bc51877cd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:44.819988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:44.858333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977733fa-5ad9-467a-bd24-80ffee4d8ac3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:44.869671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977733fa-5ad9-467a-bd24-80ffee4d8ac3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:44.873228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe253a-341b-4738-829e-07749d05a159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:44.873914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe253a-341b-4738-829e-07749d05a159 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:44.874204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe253a-341b-4738-829e-07749d05a159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:44.874485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe253a-341b-4738-829e-07749d05a159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:44.883266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe253a-341b-4738-829e-07749d05a159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:44.884699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe253a-341b-4738-829e-07749d05a159 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:44.893969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 716/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:47.060896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43ad39-2550-42bb-ad68-330440bce888 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:47.061463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43ad39-2550-42bb-ad68-330440bce888 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:47.061463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43ad39-2550-42bb-ad68-330440bce888 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:47.062870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:47.068774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4e28d-4ad7-4462-8fb6-b876200dfc0b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:47.069207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4e28d-4ad7-4462-8fb6-b876200dfc0b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:47.069339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4e28d-4ad7-4462-8fb6-b876200dfc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:47.087665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1ea4e28d-4ad7-4462-8fb6-b876200dfc0b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:47.088859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1ea4e28d-4ad7-4462-8fb6-b876200dfc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:47.096931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:36:47.134580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43ad39-2550-42bb-ad68-330440bce888 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:47.146400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43ad39-2550-42bb-ad68-330440bce888 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:47.149937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bc7c8-41dc-410e-9292-9596d432571b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:47.150579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bc7c8-41dc-410e-9292-9596d432571b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:47.150867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bc7c8-41dc-410e-9292-9596d432571b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:47.151149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bc7c8-41dc-410e-9292-9596d432571b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:47.160044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bc7c8-41dc-410e-9292-9596d432571b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:47.161473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bc7c8-41dc-410e-9292-9596d432571b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:47.170842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 741/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:49.386005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945136e-b2b2-4138-97bd-e79bfc2c494b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:49.386568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945136e-b2b2-4138-97bd-e79bfc2c494b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:49.386568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945136e-b2b2-4138-97bd-e79bfc2c494b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:49.387935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:49.393526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd5ba6-7868-42a8-824a-92e6bcff8637 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:49.393986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd5ba6-7868-42a8-824a-92e6bcff8637 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:49.394227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd5ba6-7868-42a8-824a-92e6bcff8637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:49.400494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=70 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:49.414062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a4cd5ba6-7868-42a8-824a-92e6bcff8637 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:49.415266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a4cd5ba6-7868-42a8-824a-92e6bcff8637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:49.423982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:49.461063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945136e-b2b2-4138-97bd-e79bfc2c494b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:49.467004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=73 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:49.473902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945136e-b2b2-4138-97bd-e79bfc2c494b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:49.477372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5017fd2-2def-4118-bd01-61458dc4f70a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:49.478048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017fd2-2def-4118-bd01-61458dc4f70a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:49.478326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017fd2-2def-4118-bd01-61458dc4f70a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:49.478605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017fd2-2def-4118-bd01-61458dc4f70a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:49.488432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5017fd2-2def-4118-bd01-61458dc4f70a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:49.489905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5017fd2-2def-4118-bd01-61458dc4f70a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:49.499614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 719/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:51.713691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977db48-3a28-4991-bf9e-e4a1fd4a2845 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:51.714250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977db48-3a28-4991-bf9e-e4a1fd4a2845 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:51.714250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977db48-3a28-4991-bf9e-e4a1fd4a2845 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:51.715575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:51.720901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ab47-2306-4039-808a-08cad90d3ba9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:51.721315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ab47-2306-4039-808a-08cad90d3ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:51.721474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ab47-2306-4039-808a-08cad90d3ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:51.738923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-05f5ab47-2306-4039-808a-08cad90d3ba9 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:51.740121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-05f5ab47-2306-4039-808a-08cad90d3ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:51.747904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:51.785518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977db48-3a28-4991-bf9e-e4a1fd4a2845 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:51.796261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977db48-3a28-4991-bf9e-e4a1fd4a2845 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:51.800194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fcc1d-5bf0-4b19-8f5f-d1af54237bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:51.800833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fcc1d-5bf0-4b19-8f5f-d1af54237bdd service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:51.801130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fcc1d-5bf0-4b19-8f5f-d1af54237bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:51.801406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fcc1d-5bf0-4b19-8f5f-d1af54237bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:51.810175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fcc1d-5bf0-4b19-8f5f-d1af54237bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:51.811598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fcc1d-5bf0-4b19-8f5f-d1af54237bdd service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:51.821540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 744/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:53.967454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851b2ed-748d-4b6f-b2a1-a40564287024 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:53.968022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851b2ed-748d-4b6f-b2a1-a40564287024 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:53.968022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851b2ed-748d-4b6f-b2a1-a40564287024 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:53.969456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:53.974670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42352-e71c-4c70-a43a-4fd25b3d1d65 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:53.975115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42352-e71c-4c70-a43a-4fd25b3d1d65 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:53.975369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42352-e71c-4c70-a43a-4fd25b3d1d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:53.993054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b2c42352-e71c-4c70-a43a-4fd25b3d1d65 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:53.994311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b2c42352-e71c-4c70-a43a-4fd25b3d1d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:54.002891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:54.039297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851b2ed-748d-4b6f-b2a1-a40564287024 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:54.050170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851b2ed-748d-4b6f-b2a1-a40564287024 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:54.053645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c51b2-f7db-4305-938f-a417cc223895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:54.054314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c51b2-f7db-4305-938f-a417cc223895 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:54.054605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c51b2-f7db-4305-938f-a417cc223895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:54.054891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c51b2-f7db-4305-938f-a417cc223895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:54.064216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c51b2-f7db-4305-938f-a417cc223895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:54.065640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c51b2-f7db-4305-938f-a417cc223895 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:54.074988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 722/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:36:56.214279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db73dd-dd95-4bc0-b69f-ba3e5d58d79c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:56.214825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db73dd-dd95-4bc0-b69f-ba3e5d58d79c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:56.214825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db73dd-dd95-4bc0-b69f-ba3e5d58d79c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:56.216228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:56.222175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9035c88-de21-4f57-ae5c-9d53478496e2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:56.222582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9035c88-de21-4f57-ae5c-9d53478496e2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:56.222749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9035c88-de21-4f57-ae5c-9d53478496e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:56.241569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c9035c88-de21-4f57-ae5c-9d53478496e2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:56.242826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c9035c88-de21-4f57-ae5c-9d53478496e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:56.250910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:36:56.287648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db73dd-dd95-4bc0-b69f-ba3e5d58d79c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:56.299042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db73dd-dd95-4bc0-b69f-ba3e5d58d79c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:56.302570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71fe60-9e59-434d-b5d8-897c70335d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:56.303218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71fe60-9e59-434d-b5d8-897c70335d71 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:56.303508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71fe60-9e59-434d-b5d8-897c70335d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:56.303793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71fe60-9e59-434d-b5d8-897c70335d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:56.312623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71fe60-9e59-434d-b5d8-897c70335d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:56.314096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71fe60-9e59-434d-b5d8-897c70335d71 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:56.323359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 747/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:06.235882 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=17 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:06.319254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=17 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:16.657588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ceda9-1fe4-4359-8e7f-3042c661c2e6 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.658185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ceda9-1fe4-4359-8e7f-3042c661c2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.658185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ceda9-1fe4-4359-8e7f-3042c661c2e6 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.659572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 724/1471] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:16.664857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7c77-2c98-4082-9225-8b911b66bb4d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.665257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7c77-2c98-4082-9225-8b911b66bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.665417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7c77-2c98-4082-9225-8b911b66bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.683175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3a3e7c77-2c98-4082-9225-8b911b66bb4d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:16.684378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3a3e7c77-2c98-4082-9225-8b911b66bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:16.692765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 748/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:16.741477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce0d5e-202d-402b-a66f-2c271dcee280 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.741806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce0d5e-202d-402b-a66f-2c271dcee280 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.742088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce0d5e-202d-402b-a66f-2c271dcee280 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.743483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 725/1473] 10.222.0.57 () {54 vars in 962 bytes} [Wed Jun 15 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:16.749839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6715-90fd-4676-ad5c-846309f025dd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.750141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6715-90fd-4676-ad5c-846309f025dd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.750429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6715-90fd-4676-ad5c-846309f025dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.768182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-aeea6715-90fd-4676-ad5c-846309f025dd None None] MFA Rules not processed for user `3ca2cd515aa84a3ca01656977036ac07`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:16.769371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-aeea6715-90fd-4676-ad5c-846309f025dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:16.775753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 749/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Jun 15 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:16.783648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce0d5e-202d-402b-a66f-2c271dcee280 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:16.794411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce0d5e-202d-402b-a66f-2c271dcee280 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:16.798762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce7fe1-48ae-4fcc-9bff-533c43239c4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:16.799443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce7fe1-48ae-4fcc-9bff-533c43239c4d service glance] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.799735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce7fe1-48ae-4fcc-9bff-533c43239c4d service glance] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.800021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce7fe1-48ae-4fcc-9bff-533c43239c4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.809215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce7fe1-48ae-4fcc-9bff-533c43239c4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:16.810669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce7fe1-48ae-4fcc-9bff-533c43239c4d service glance] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:16.820833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 726/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:19.035535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceca89a-c8a6-464a-82e0-558e6c08caa3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.036112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceca89a-c8a6-464a-82e0-558e6c08caa3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.036112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceca89a-c8a6-464a-82e0-558e6c08caa3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.037469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 750/1476] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:19.042712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49ad20-36d0-4eaa-b3d9-6364c508d099 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.043115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49ad20-36d0-4eaa-b3d9-6364c508d099 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.043314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49ad20-36d0-4eaa-b3d9-6364c508d099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.060791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0c49ad20-36d0-4eaa-b3d9-6364c508d099 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:19.062215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c49ad20-36d0-4eaa-b3d9-6364c508d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:19.070260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 727/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:19.095214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceca89a-c8a6-464a-82e0-558e6c08caa3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:19.107281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceca89a-c8a6-464a-82e0-558e6c08caa3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:19.111867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:19.112509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.112794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.113072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.113835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:19.115605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:37:19.115605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:37:19.115605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:37:19.116743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e69270-71ce-40b5-b533-53b774477b0d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:19.135539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 751/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Jun 15 14:37:19 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:37:21.299633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0c83-9ff6-46dc-bbee-f1683f8920fc None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:21.300201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0c83-9ff6-46dc-bbee-f1683f8920fc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:21.300329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0c83-9ff6-46dc-bbee-f1683f8920fc None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:21.301647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 728/1479] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:21.306990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a702-504a-4045-bf8b-a54b414e62b6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:21.307427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a702-504a-4045-bf8b-a54b414e62b6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:21.307565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a702-504a-4045-bf8b-a54b414e62b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:21.325914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0f80a702-504a-4045-bf8b-a54b414e62b6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:21.327324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f80a702-504a-4045-bf8b-a54b414e62b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:21.335235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 752/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:21.379949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0c83-9ff6-46dc-bbee-f1683f8920fc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:21.395247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0c83-9ff6-46dc-bbee-f1683f8920fc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:21.400691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6359b3-af63-4418-92b7-77716c8c0ad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:21.401338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6359b3-af63-4418-92b7-77716c8c0ad0 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:21.401630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6359b3-af63-4418-92b7-77716c8c0ad0 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:21.401945 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6359b3-af63-4418-92b7-77716c8c0ad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:21.410108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6359b3-af63-4418-92b7-77716c8c0ad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:21.411682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6359b3-af63-4418-92b7-77716c8c0ad0 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:21.421595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 729/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:23.746972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b91d0-a615-492e-81e2-edf7dd647a26 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.747504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b91d0-a615-492e-81e2-edf7dd647a26 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.747504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b91d0-a615-492e-81e2-edf7dd647a26 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.749336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 753/1482] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:23.754754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9424f-2c5e-42c2-9d61-755ca551efe0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.755193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9424f-2c5e-42c2-9d61-755ca551efe0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.755329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9424f-2c5e-42c2-9d61-755ca551efe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.773469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e3e9424f-2c5e-42c2-9d61-755ca551efe0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:23.774712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e3e9424f-2c5e-42c2-9d61-755ca551efe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:23.782844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 730/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:23.818635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b91d0-a615-492e-81e2-edf7dd647a26 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:23.829955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b91d0-a615-492e-81e2-edf7dd647a26 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:23.833414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0812dcb-6784-4f10-8470-7b0fa7b9e0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:23.834090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812dcb-6784-4f10-8470-7b0fa7b9e0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.834381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812dcb-6784-4f10-8470-7b0fa7b9e0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.834662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0812dcb-6784-4f10-8470-7b0fa7b9e0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.845066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0812dcb-6784-4f10-8470-7b0fa7b9e0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:23.846540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0812dcb-6784-4f10-8470-7b0fa7b9e0d2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:23.855991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 754/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:26.037552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21b5fb-6227-4889-84eb-5f2ff05012c4 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.038160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21b5fb-6227-4889-84eb-5f2ff05012c4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.038160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21b5fb-6227-4889-84eb-5f2ff05012c4 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.039582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 731/1485] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:26.044732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893ef54-6019-4d48-8002-9ac328db3ab0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.045130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893ef54-6019-4d48-8002-9ac328db3ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.045367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893ef54-6019-4d48-8002-9ac328db3ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.063683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e893ef54-6019-4d48-8002-9ac328db3ab0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:26.064878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e893ef54-6019-4d48-8002-9ac328db3ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:26.072616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 755/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:26.111683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21b5fb-6227-4889-84eb-5f2ff05012c4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:26.122369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21b5fb-6227-4889-84eb-5f2ff05012c4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:26.126105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5d539-9606-4fd1-ba59-e68d9a58aa98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:26.126742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5d539-9606-4fd1-ba59-e68d9a58aa98 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.127033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5d539-9606-4fd1-ba59-e68d9a58aa98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.127320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5d539-9606-4fd1-ba59-e68d9a58aa98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.136248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5d539-9606-4fd1-ba59-e68d9a58aa98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:26.137698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5d539-9606-4fd1-ba59-e68d9a58aa98 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:26.147083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 732/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:28.443717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c47725-d4fa-42da-b7e1-e1f4f40c4b2d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:28.444415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c47725-d4fa-42da-b7e1-e1f4f40c4b2d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:28.444518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c47725-d4fa-42da-b7e1-e1f4f40c4b2d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:28.445912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 756/1488] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:28.451286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24779-be6d-45f9-b9fa-1a5c35197de0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:28.451853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24779-be6d-45f9-b9fa-1a5c35197de0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:28.452000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24779-be6d-45f9-b9fa-1a5c35197de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:28.470586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-66c24779-be6d-45f9-b9fa-1a5c35197de0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:28.471767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-66c24779-be6d-45f9-b9fa-1a5c35197de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:28.479561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 733/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:37:28.520342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c47725-d4fa-42da-b7e1-e1f4f40c4b2d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:28.532169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c47725-d4fa-42da-b7e1-e1f4f40c4b2d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:28.535688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf7ee6-a868-4867-8330-b3263f1e98fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:28.536335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf7ee6-a868-4867-8330-b3263f1e98fb service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:28.536624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf7ee6-a868-4867-8330-b3263f1e98fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:28.536905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf7ee6-a868-4867-8330-b3263f1e98fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:28.546296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf7ee6-a868-4867-8330-b3263f1e98fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:28.547667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf7ee6-a868-4867-8330-b3263f1e98fb service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:28.556913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 757/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:30.834411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f56e-c29c-4979-b052-2efd8828608f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:30.834978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f56e-c29c-4979-b052-2efd8828608f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:30.834978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f56e-c29c-4979-b052-2efd8828608f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:30.836621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 734/1491] 10.222.0.57 () {54 vars in 926 bytes} [Wed Jun 15 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:30.841773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408e6-dbd1-4dc2-bf32-414bdef5b468 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:30.842228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408e6-dbd1-4dc2-bf32-414bdef5b468 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:30.842416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408e6-dbd1-4dc2-bf32-414bdef5b468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:30.861346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2ff408e6-dbd1-4dc2-bf32-414bdef5b468 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:30.862591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ff408e6-dbd1-4dc2-bf32-414bdef5b468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:30.870342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 758/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:37:30.908777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6f56e-c29c-4979-b052-2efd8828608f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:30.919429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6f56e-c29c-4979-b052-2efd8828608f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:30.923701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56666598-db5c-4033-9a2d-e32d483365d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:30.924342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56666598-db5c-4033-9a2d-e32d483365d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:30.924633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56666598-db5c-4033-9a2d-e32d483365d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:30.924922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56666598-db5c-4033-9a2d-e32d483365d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:30.933900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56666598-db5c-4033-9a2d-e32d483365d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:30.935321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56666598-db5c-4033-9a2d-e32d483365d5 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:30.945550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 735/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:37:40.854714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=33 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:40.941320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=33 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:56.075237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.075836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.075836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.094349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:56.097173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:56.105577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 759/1494] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:56.114780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.115291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.115397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.134014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:56.135470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:56.143228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 736/1495] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:56.147786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.158225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d318dad-8337-45b4-b58c-8fdef6169893 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.162718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.163372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.163659 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.163935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.164657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:56.164969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.166668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.166668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:38:56.166668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:38:56.167553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.178897 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 760/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:56.187172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.196430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4dfc28-fd08-4e03-8837-53c527535878 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.200953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.201633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.201948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.202230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.202978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.204765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.204765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:38:56.204765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:38:56.206000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.242100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 737/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:38:56.247007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.257350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee2e1e-c887-4f1a-b5c4-145f49061d16 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.312226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.312902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.313194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.313477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.314205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.315221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.315221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:38:56.315221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:38:56.316427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.337345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 761/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:38:56.341559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.351842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631db027-726e-474b-90ee-84addb21ef05 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.357057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.357751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.358034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.358311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.359067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.360189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.360189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:38:56.360189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:38:56.361067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.374363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 738/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:38:56.378554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.388503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3d11-f5e6-4a8b-ba6e-68afb24e507d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.392954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.393615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.393932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.394215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.394947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.395953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.395953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:38:56.395953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:38:56.396834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f513bc-9e7b-40e4-a351-531914b13ffa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.410112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 762/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:38:56.414214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.424530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d7328-d32f-41a2-9d7e-a4704b8f9b16 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.429115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.430035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.430324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.430608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] PATH_INFO: `/v3/projects/6f02b1f393b646459b534c1823dc9531/users/012f276a84eb42d89ce254c0397b2794/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.445850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f02b1f393b646459b534c1823dc9531, user_id=012f276a84eb42d89ce254c0397b2794, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.446936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.446936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:38:56.446936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:38:56.448869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c517bc-08ba-4908-af77-5ec51783cc7a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.462716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 739/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:38:56 2022] PUT /identity/v3/projects/6f02b1f393b646459b534c1823dc9531/users/012f276a84eb42d89ce254c0397b2794/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:38:56.474134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf62ad2-06f3-48bb-9e1f-cfd822e0877f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.474430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf62ad2-06f3-48bb-9e1f-cfd822e0877f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.474715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf62ad2-06f3-48bb-9e1f-cfd822e0877f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.501394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-adf62ad2-06f3-48bb-9e1f-cfd822e0877f None None] MFA Rules not processed for user `012f276a84eb42d89ce254c0397b2794`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:56.502632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-adf62ad2-06f3-48bb-9e1f-cfd822e0877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:56.565908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 763/1502] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:56.579219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48325-11a2-43c0-bd28-9d5bd547b9e2 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.579516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48325-11a2-43c0-bd28-9d5bd547b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.579802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48325-11a2-43c0-bd28-9d5bd547b9e2 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.581218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 740/1503] 10.222.0.57 () {54 vars in 963 bytes} [Wed Jun 15 14:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:38:56.586582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31032d01-205c-4fd6-911d-59b35d5e5a0b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.586877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31032d01-205c-4fd6-911d-59b35d5e5a0b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.587215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31032d01-205c-4fd6-911d-59b35d5e5a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.605474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-31032d01-205c-4fd6-911d-59b35d5e5a0b None None] MFA Rules not processed for user `a26d10743fe844a98ecfcae3974dbce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:56.606639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-31032d01-205c-4fd6-911d-59b35d5e5a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:56.649522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:56.659350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 764/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:56.667846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48325-11a2-43c0-bd28-9d5bd547b9e2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.678444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48325-11a2-43c0-bd28-9d5bd547b9e2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.682611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.683254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.683544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.683828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.692029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.693437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.702396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 741/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:38:56.745069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80932b-7a30-427d-bc53-83361c8bc650 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.745366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80932b-7a30-427d-bc53-83361c8bc650 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.745647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80932b-7a30-427d-bc53-83361c8bc650 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.746978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 765/1506] 10.222.0.57 () {52 vars in 829 bytes} [Wed Jun 15 14:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:38:56.750626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.812167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48390b2-ee1f-4ad2-8850-6bfaeb2b6b44 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.816704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.817577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.817897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.818183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] PATH_INFO: `/v3/users/012f276a84eb42d89ce254c0397b2794` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.819369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=012f276a84eb42d89ce254c0397b2794)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.820382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:56.820382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:38:56.820382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:38:56.821726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.886299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012f276a84eb42d89ce254c0397b2794'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:56.904846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012f276a84eb42d89ce254c0397b2794'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:56.918303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012f276a84eb42d89ce254c0397b2794'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:56.922458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:56.929022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012f276a84eb42d89ce254c0397b2794'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:56.929863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012f276a84eb42d89ce254c0397b2794'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:56.937566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 742/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:38:56 2022] DELETE /identity/v3/users/012f276a84eb42d89ce254c0397b2794 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:38:56.942076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.942376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.942689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.959963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:56.961166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:57.018281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 766/1508] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:38:57.028166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.029591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:57.086357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0d9b5-280b-454d-901a-8b5110cfca77 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.091014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d16b1-389f-4d37-af5f-b0c12bcb0d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.091666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d16b1-389f-4d37-af5f-b0c12bcb0d5c service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.091951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d16b1-389f-4d37-af5f-b0c12bcb0d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.092232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d16b1-389f-4d37-af5f-b0c12bcb0d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.101108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d16b1-389f-4d37-af5f-b0c12bcb0d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.102552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d16b1-389f-4d37-af5f-b0c12bcb0d5c service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.111950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 743/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:38:57.438778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.440387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:57.453936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa43b-7c9f-4fe3-b74e-ab86a05dd710 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.458994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.459829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.460225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.460397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] PATH_INFO: `/v3/projects/6f02b1f393b646459b534c1823dc9531` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.461621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f02b1f393b646459b534c1823dc9531)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.463455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:38:57.463455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:38:57.463455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:38:57.464667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.528735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f02b1f393b646459b534c1823dc9531', 'request_id': 'req-0bbbf842-dd71-4c62-a424-7661c03958f8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:57.538185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f02b1f393b646459b534c1823dc9531', 'request_id': 'req-0bbbf842-dd71-4c62-a424-7661c03958f8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:57.544459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f02b1f393b646459b534c1823dc9531', 'request_id': 'req-0bbbf842-dd71-4c62-a424-7661c03958f8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:57.564512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] The token cache is being invalidate because project 6f02b1f393b646459b534c1823dc9531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:38:57.564804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:57.565456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0bbbf842-dd71-4c62-a424-7661c03958f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:38:57.567115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 767/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:38:57 2022] DELETE /identity/v3/projects/6f02b1f393b646459b534c1823dc9531 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:39:07.107831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:07.561402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:40:43.963188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969ded-9353-42e2-88e8-f6319b9ae647 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:43.963791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969ded-9353-42e2-88e8-f6319b9ae647 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:43.963791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969ded-9353-42e2-88e8-f6319b9ae647 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:43.965135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:43.969978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b57d-29ad-4630-b6fc-730a1d05c5be None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:43.970427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b57d-29ad-4630-b6fc-730a1d05c5be None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:43.970533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b57d-29ad-4630-b6fc-730a1d05c5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:43.989150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0d57b57d-29ad-4630-b6fc-730a1d05c5be None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:43.990391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0d57b57d-29ad-4630-b6fc-730a1d05c5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:44.050111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:44.222818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969ded-9353-42e2-88e8-f6319b9ae647 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:44.225092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eb969ded-9353-42e2-88e8-f6319b9ae647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:44.299798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969ded-9353-42e2-88e8-f6319b9ae647 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:44.305127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2c8be-5012-4990-aa8f-45fb53e498c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:44.305809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2c8be-5012-4990-aa8f-45fb53e498c1 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:44.306096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2c8be-5012-4990-aa8f-45fb53e498c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:44.306379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2c8be-5012-4990-aa8f-45fb53e498c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:44.314754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2c8be-5012-4990-aa8f-45fb53e498c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:44.316365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2c8be-5012-4990-aa8f-45fb53e498c1 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:44.329010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 745/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:45.207252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19eb06-d14f-4105-924f-4559fe674f8c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:45.208073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19eb06-d14f-4105-924f-4559fe674f8c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:45.208073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19eb06-d14f-4105-924f-4559fe674f8c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:45.209466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:45.214362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee546a-5729-4a87-8f50-db7f10a48239 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:45.214799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee546a-5729-4a87-8f50-db7f10a48239 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:45.214934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee546a-5729-4a87-8f50-db7f10a48239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:45.232966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7eee546a-5729-4a87-8f50-db7f10a48239 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:45.234188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7eee546a-5729-4a87-8f50-db7f10a48239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:45.241981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:45.400394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19eb06-d14f-4105-924f-4559fe674f8c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:45.414973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19eb06-d14f-4105-924f-4559fe674f8c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:45.420973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153e7ab-a499-43ba-b42e-688c211bc76f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:45.421608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153e7ab-a499-43ba-b42e-688c211bc76f service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:45.421922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153e7ab-a499-43ba-b42e-688c211bc76f service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:45.422201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153e7ab-a499-43ba-b42e-688c211bc76f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:45.430171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153e7ab-a499-43ba-b42e-688c211bc76f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:45.431748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153e7ab-a499-43ba-b42e-688c211bc76f service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:45.441948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 770/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:48.411049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812db63f-f069-4184-85aa-2a71011baf12 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:48.411652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812db63f-f069-4184-85aa-2a71011baf12 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:48.411652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812db63f-f069-4184-85aa-2a71011baf12 None None] PATH_INFO: `/v3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:48.412911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 747/1517] 10.222.0.70 () {54 vars in 925 bytes} [Wed Jun 15 14:40:48 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:40:48.422722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e301e-bcc6-4c00-9f75-9c23b19130f9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:48.423189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e301e-bcc6-4c00-9f75-9c23b19130f9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:48.423298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e301e-bcc6-4c00-9f75-9c23b19130f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:48.441813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bb2e301e-bcc6-4c00-9f75-9c23b19130f9 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:48.443005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bb2e301e-bcc6-4c00-9f75-9c23b19130f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:48.507745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 771/1518] 10.222.0.70 () {58 vars in 1015 bytes} [Wed Jun 15 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:55.227846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=37 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:40:56.701940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e75a0-46a2-4350-a2e4-cbdf3441450e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:56.702599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e75a0-46a2-4350-a2e4-cbdf3441450e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:56.702599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e75a0-46a2-4350-a2e4-cbdf3441450e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:56.704034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 748/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:40:56.709364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd70a-b0b9-43ba-a37b-00726856e5e3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:56.709840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd70a-b0b9-43ba-a37b-00726856e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:56.709947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd70a-b0b9-43ba-a37b-00726856e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:56.727769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fcbdd70a-b0b9-43ba-a37b-00726856e5e3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:56.728988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fcbdd70a-b0b9-43ba-a37b-00726856e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:56.737421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 772/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:40:56.893301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e75a0-46a2-4350-a2e4-cbdf3441450e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:56.907917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e75a0-46a2-4350-a2e4-cbdf3441450e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:56.913233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8762fae-7916-4afe-820d-3628066ecfa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:56.913945 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762fae-7916-4afe-820d-3628066ecfa9 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:56.914232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762fae-7916-4afe-820d-3628066ecfa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:56.914514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762fae-7916-4afe-820d-3628066ecfa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:56.923300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8762fae-7916-4afe-820d-3628066ecfa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:56.924875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8762fae-7916-4afe-820d-3628066ecfa9 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:56.934939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 749/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:40:57.808539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08262e78-9072-472b-9d92-2825a4e80b11 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:57.809107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08262e78-9072-472b-9d92-2825a4e80b11 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:57.809528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08262e78-9072-472b-9d92-2825a4e80b11 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:57.810998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:40:57.815853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fbd5e-6c20-4c46-9558-01472a168941 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:57.816298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fbd5e-6c20-4c46-9558-01472a168941 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:57.816415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fbd5e-6c20-4c46-9558-01472a168941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:57.837096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c16fbd5e-6c20-4c46-9558-01472a168941 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:40:57.838394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c16fbd5e-6c20-4c46-9558-01472a168941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:40:57.847061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:40:57.998406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08262e78-9072-472b-9d92-2825a4e80b11 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:40:58.012904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08262e78-9072-472b-9d92-2825a4e80b11 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:40:58.018622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b37cb2-d410-4951-aa65-3f93807a2e5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:40:58.019272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b37cb2-d410-4951-aa65-3f93807a2e5f service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:40:58.019563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b37cb2-d410-4951-aa65-3f93807a2e5f service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:40:58.019847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b37cb2-d410-4951-aa65-3f93807a2e5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:40:58.028927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b37cb2-d410-4951-aa65-3f93807a2e5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:40:58.030569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b37cb2-d410-4951-aa65-3f93807a2e5f service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:40:58.039993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 774/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:40:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:00.155139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937dcd0-5d0e-4a63-a7d3-92c5e9d5f01b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:00.155906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937dcd0-5d0e-4a63-a7d3-92c5e9d5f01b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:00.156082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937dcd0-5d0e-4a63-a7d3-92c5e9d5f01b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:00.157487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 751/1525] 10.222.0.57 () {54 vars in 960 bytes} [Wed Jun 15 14:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:00.162514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc729-7e4f-4049-ac3d-0de8d6b6af54 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:00.162953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc729-7e4f-4049-ac3d-0de8d6b6af54 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:00.163088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc729-7e4f-4049-ac3d-0de8d6b6af54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:00.180612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1cdfc729-7e4f-4049-ac3d-0de8d6b6af54 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:00.181838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1cdfc729-7e4f-4049-ac3d-0de8d6b6af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:00.190837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 775/1526] 10.222.0.57 () {58 vars in 1057 bytes} [Wed Jun 15 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:00.198082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937dcd0-5d0e-4a63-a7d3-92c5e9d5f01b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:00.212440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937dcd0-5d0e-4a63-a7d3-92c5e9d5f01b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:00.217620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228513d-f313-4d24-87ec-56f8ad34dd04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:00.218312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228513d-f313-4d24-87ec-56f8ad34dd04 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:00.218598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228513d-f313-4d24-87ec-56f8ad34dd04 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:00.218877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228513d-f313-4d24-87ec-56f8ad34dd04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:00.227057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228513d-f313-4d24-87ec-56f8ad34dd04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:00.228648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228513d-f313-4d24-87ec-56f8ad34dd04 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:00.238708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 752/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:41:09.327534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ce51-a0e6-436f-b7ba-53c2337d036a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:09.328141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ce51-a0e6-436f-b7ba-53c2337d036a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:09.328264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ce51-a0e6-436f-b7ba-53c2337d036a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:09.329683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:41:09.334722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a9ee-28cb-489f-951d-3eaeefaa6646 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:09.335172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a9ee-28cb-489f-951d-3eaeefaa6646 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:09.335318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a9ee-28cb-489f-951d-3eaeefaa6646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:09.353138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e9b4a9ee-28cb-489f-951d-3eaeefaa6646 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:09.354348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e9b4a9ee-28cb-489f-951d-3eaeefaa6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:09.363013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:09.523336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ce51-a0e6-436f-b7ba-53c2337d036a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:09.531937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=75 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:09.539426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ce51-a0e6-436f-b7ba-53c2337d036a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:09.545845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55a66f-96b5-4136-85fa-7be647cc5c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:09.546546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a66f-96b5-4136-85fa-7be647cc5c2e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:09.546857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a66f-96b5-4136-85fa-7be647cc5c2e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:09.547164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a66f-96b5-4136-85fa-7be647cc5c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:09.555965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a66f-96b5-4136-85fa-7be647cc5c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:09.557696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a66f-96b5-4136-85fa-7be647cc5c2e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:09.568242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 777/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:10.432734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454406b-e0be-42c5-98cf-8c3d41188c51 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:10.433351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454406b-e0be-42c5-98cf-8c3d41188c51 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:10.433351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454406b-e0be-42c5-98cf-8c3d41188c51 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:10.435499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:10.440367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98941e7e-aea5-4a4c-be78-3f847db1f7d6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:10.440827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98941e7e-aea5-4a4c-be78-3f847db1f7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:10.441016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98941e7e-aea5-4a4c-be78-3f847db1f7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:10.460272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-98941e7e-aea5-4a4c-be78-3f847db1f7d6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:10.461511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98941e7e-aea5-4a4c-be78-3f847db1f7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:10.469565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:10.623412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454406b-e0be-42c5-98cf-8c3d41188c51 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:10.638330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454406b-e0be-42c5-98cf-8c3d41188c51 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:10.644154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe1ae5-1346-4861-b79f-50d2f468ef7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:10.644796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1ae5-1346-4861-b79f-50d2f468ef7f service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:10.645085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1ae5-1346-4861-b79f-50d2f468ef7f service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:10.645368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1ae5-1346-4861-b79f-50d2f468ef7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:10.653640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe1ae5-1346-4861-b79f-50d2f468ef7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:10.655235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe1ae5-1346-4861-b79f-50d2f468ef7f service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:10.664547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 755/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:15.550543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb24c36-6b50-4c7a-8a78-88d53ef7d739 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:15.551120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb24c36-6b50-4c7a-8a78-88d53ef7d739 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:15.551120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb24c36-6b50-4c7a-8a78-88d53ef7d739 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:15.552470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:15.557531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e9763-6df0-4100-818a-d41e3200a84e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:15.557987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e9763-6df0-4100-818a-d41e3200a84e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:15.558135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e9763-6df0-4100-818a-d41e3200a84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:15.576424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eb9e9763-6df0-4100-818a-d41e3200a84e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:15.577644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eb9e9763-6df0-4100-818a-d41e3200a84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:15.585687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:15.623820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb24c36-6b50-4c7a-8a78-88d53ef7d739 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:15.625508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1bb24c36-6b50-4c7a-8a78-88d53ef7d739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:15.682404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb24c36-6b50-4c7a-8a78-88d53ef7d739 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:15.686669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e858a8a-31e0-436f-8b88-b40b844d8ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:15.687315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e858a8a-31e0-436f-8b88-b40b844d8ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:15.687606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e858a8a-31e0-436f-8b88-b40b844d8ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:15.687889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e858a8a-31e0-436f-8b88-b40b844d8ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:15.696705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e858a8a-31e0-436f-8b88-b40b844d8ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:15.698176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e858a8a-31e0-436f-8b88-b40b844d8ea1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:15.708035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 780/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:41:17.827818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea0873-8c5d-499a-a58d-548e38a28a96 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:17.828381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea0873-8c5d-499a-a58d-548e38a28a96 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:17.828381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea0873-8c5d-499a-a58d-548e38a28a96 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:17.829835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:17.835125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4120daf-6181-44cb-9e6e-cd81544830d0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:17.835518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4120daf-6181-44cb-9e6e-cd81544830d0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:17.835692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4120daf-6181-44cb-9e6e-cd81544830d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:17.853633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d4120daf-6181-44cb-9e6e-cd81544830d0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:17.855039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d4120daf-6181-44cb-9e6e-cd81544830d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:17.863585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:17.900297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea0873-8c5d-499a-a58d-548e38a28a96 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:17.902059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-33ea0873-8c5d-499a-a58d-548e38a28a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:17.915268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea0873-8c5d-499a-a58d-548e38a28a96 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:17.919358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ffb4e-81aa-45f4-b9f9-a90d773eca01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:17.920141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ffb4e-81aa-45f4-b9f9-a90d773eca01 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:17.920433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ffb4e-81aa-45f4-b9f9-a90d773eca01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:17.920718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ffb4e-81aa-45f4-b9f9-a90d773eca01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:17.929561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ffb4e-81aa-45f4-b9f9-a90d773eca01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:17.931007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ffb4e-81aa-45f4-b9f9-a90d773eca01 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:17.941153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 758/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:21.710661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3d1b2-5805-4d11-8637-f8c7bc7e1ab0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:21.711136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3d1b2-5805-4d11-8637-f8c7bc7e1ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:21.711233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3d1b2-5805-4d11-8637-f8c7bc7e1ab0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:21.712632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 782/1540] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:21.718614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7830b81-d873-4ce7-af27-e6e5833a0751 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:21.719064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7830b81-d873-4ce7-af27-e6e5833a0751 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:21.719172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7830b81-d873-4ce7-af27-e6e5833a0751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:21.737837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e7830b81-d873-4ce7-af27-e6e5833a0751 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:21.738969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e7830b81-d873-4ce7-af27-e6e5833a0751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:21.746740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 759/1541] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:22.012989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3d1b2-5805-4d11-8637-f8c7bc7e1ab0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:22.015083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-04d3d1b2-5805-4d11-8637-f8c7bc7e1ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:22.072463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3d1b2-5805-4d11-8637-f8c7bc7e1ab0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:22.076519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0ee4-6fa2-4ee9-af59-60c1a5f74169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:22.077168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0ee4-6fa2-4ee9-af59-60c1a5f74169 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:22.077456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0ee4-6fa2-4ee9-af59-60c1a5f74169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:22.077766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0ee4-6fa2-4ee9-af59-60c1a5f74169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:22.086781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b0ee4-6fa2-4ee9-af59-60c1a5f74169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:22.088209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b0ee4-6fa2-4ee9-af59-60c1a5f74169 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:22.098149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 783/1542] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:24.274422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d23-dc73-4622-bf0b-8247337aceab None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:24.275002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d23-dc73-4622-bf0b-8247337aceab None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:24.275002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d23-dc73-4622-bf0b-8247337aceab None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:24.276447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 760/1543] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:41:24.282399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d94e0-91e2-457b-bd29-84dc87c64b1f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:24.282817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d94e0-91e2-457b-bd29-84dc87c64b1f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:24.282973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d94e0-91e2-457b-bd29-84dc87c64b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:24.302006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f89d94e0-91e2-457b-bd29-84dc87c64b1f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:41:24.303218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f89d94e0-91e2-457b-bd29-84dc87c64b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:24.310972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 784/1544] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:41:24.579308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b257d23-dc73-4622-bf0b-8247337aceab None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:41:24.581031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0b257d23-dc73-4622-bf0b-8247337aceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:41:24.594261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b257d23-dc73-4622-bf0b-8247337aceab None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:41:24.597829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf171ecb-422f-4ad4-985c-7a6e690f5896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:41:24.598470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf171ecb-422f-4ad4-985c-7a6e690f5896 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:41:24.598759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf171ecb-422f-4ad4-985c-7a6e690f5896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:41:24.599041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf171ecb-422f-4ad4-985c-7a6e690f5896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:41:24.608409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf171ecb-422f-4ad4-985c-7a6e690f5896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:41:24.609861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf171ecb-422f-4ad4-985c-7a6e690f5896 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:41:24.619207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 761/1545] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:41:34.296604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=66 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:41:34.615071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=42 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:58.607167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.607769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.607912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.631994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:58.633864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:58.650195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 785/1546] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:58.660827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.661866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.662388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.684274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:58.686292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:58.698597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 762/1547] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:58.703235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:58.714448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8fb9f-ce78-4c61-875f-4acda418e5c4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:58.727526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:58.728547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.729063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.729602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.730593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:58.731138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:58.733865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:58.751095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 786/1548] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:58.760795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:58.774963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4aad-e20a-4313-bdcf-35331b6f7d06 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:58.782116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:58.783131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.783646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.784148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.785097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:58.787275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:58.830902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 763/1549] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:58 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:58.835439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:58.847149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8093-1233-4320-b94d-b73da65635e0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:58.925350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:58.926567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:58.927094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:58.927596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:58.928571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:58.930782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:58.972313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 787/1550] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:58 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:58.976571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:58.990953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889c628-e8ab-45ea-83ce-e35d4b6bd566 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.002275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.003328 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.003867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.004360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.005308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.007167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.024197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 764/1551] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:59.028196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.042043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d237d2a-5356-4d74-bb76-8572833997fd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.050320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa0037-d85f-4a27-a9f6-a48c8b8a368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.051364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0037-d85f-4a27-a9f6-a48c8b8a368a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.051861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0037-d85f-4a27-a9f6-a48c8b8a368a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.052352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0037-d85f-4a27-a9f6-a48c8b8a368a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.053432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa0037-d85f-4a27-a9f6-a48c8b8a368a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.055304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa0037-d85f-4a27-a9f6-a48c8b8a368a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.071693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 788/1552] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:42:59.076572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.093453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bbf79-bd78-4da5-889e-e7d5e2118f71 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.103997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c69c84-a706-4efe-bf53-0c3b110d441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.105013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c69c84-a706-4efe-bf53-0c3b110d441c admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.105310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c69c84-a706-4efe-bf53-0c3b110d441c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.105600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c69c84-a706-4efe-bf53-0c3b110d441c admin admin] PATH_INFO: `/v3/projects/658ef56d424e4381a1fba60ebedf0277/users/4735488a206c4a2da7873c310a95a91e/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.127171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c69c84-a706-4efe-bf53-0c3b110d441c admin admin] RBAC: Authorizing `identity:create_grant(project_id=658ef56d424e4381a1fba60ebedf0277, user_id=4735488a206c4a2da7873c310a95a91e, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.130559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c69c84-a706-4efe-bf53-0c3b110d441c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.164305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 765/1553] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:42:59 2022] PUT /identity/v3/projects/658ef56d424e4381a1fba60ebedf0277/users/4735488a206c4a2da7873c310a95a91e/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:42:59.184372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b495a-144b-4946-8905-e3292f1e1d11 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.185022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b495a-144b-4946-8905-e3292f1e1d11 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.186390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b495a-144b-4946-8905-e3292f1e1d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.222241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-218b495a-144b-4946-8905-e3292f1e1d11 None None] MFA Rules not processed for user `4735488a206c4a2da7873c310a95a91e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.223849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-218b495a-144b-4946-8905-e3292f1e1d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.283717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4e3a-e700-4980-ac91-60ddfb4f7997 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.284599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4e3a-e700-4980-ac91-60ddfb4f7997 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.284907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4e3a-e700-4980-ac91-60ddfb4f7997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.327619 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-864f4e3a-e700-4980-ac91-60ddfb4f7997 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.328264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 789/1554] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.329542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-864f4e3a-e700-4980-ac91-60ddfb4f7997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.330379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907c72-1966-4d57-bf38-ddc3a6f46f80 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.331003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907c72-1966-4d57-bf38-ddc3a6f46f80 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.331510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907c72-1966-4d57-bf38-ddc3a6f46f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.369553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6f907c72-1966-4d57-bf38-ddc3a6f46f80 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.371112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6f907c72-1966-4d57-bf38-ddc3a6f46f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.439395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 766/1555] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:42:59.440077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f12f-f4ba-48ce-b4fd-3e354aae1f77 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.440623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f12f-f4ba-48ce-b4fd-3e354aae1f77 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.441164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f12f-f4ba-48ce-b4fd-3e354aae1f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.481048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c0c2f12f-f4ba-48ce-b4fd-3e354aae1f77 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.485748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c0c2f12f-f4ba-48ce-b4fd-3e354aae1f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.491593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:59.499528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 767/1556] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.501064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b68fe-7fa8-4e9f-b1e3-2c0949217b83 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.501357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b68fe-7fa8-4e9f-b1e3-2c0949217b83 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.501638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b68fe-7fa8-4e9f-b1e3-2c0949217b83 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.504721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 768/1557] 10.222.0.57 () {54 vars in 961 bytes} [Wed Jun 15 14:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:42:59.505917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca12d95-3b9b-4af7-b5a8-1ec710efafbb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.506215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca12d95-3b9b-4af7-b5a8-1ec710efafbb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.506494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca12d95-3b9b-4af7-b5a8-1ec710efafbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.512212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 790/1558] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.514635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719fc86-20d3-40bd-864d-9e6b96997629 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.514939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719fc86-20d3-40bd-864d-9e6b96997629 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.515227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719fc86-20d3-40bd-864d-9e6b96997629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.534782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8ca12d95-3b9b-4af7-b5a8-1ec710efafbb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.535249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4719fc86-20d3-40bd-864d-9e6b96997629 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.535987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8ca12d95-3b9b-4af7-b5a8-1ec710efafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.536426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4719fc86-20d3-40bd-864d-9e6b96997629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.545156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 769/1559] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.545416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 791/1560] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.550330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31817ce-f49b-4166-bb52-1410b0bf6a84 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.550569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.550829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.551124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.551614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31817ce-f49b-4166-bb52-1410b0bf6a84 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.552118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31817ce-f49b-4166-bb52-1410b0bf6a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.571354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.572311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f31817ce-f49b-4166-bb52-1410b0bf6a84 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.572697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.573493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f31817ce-f49b-4166-bb52-1410b0bf6a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.582056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 770/1561] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.583305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a86c8e-e02f-4361-8ce0-552bc9f61c7c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.583597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a86c8e-e02f-4361-8ce0-552bc9f61c7c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.583880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a86c8e-e02f-4361-8ce0-552bc9f61c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.602159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-28a86c8e-e02f-4361-8ce0-552bc9f61c7c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.603357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-28a86c8e-e02f-4361-8ce0-552bc9f61c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.611726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 771/1562] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.612956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c20e0b-e8fd-428e-9630-c69f2eea52e3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.613253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c20e0b-e8fd-428e-9630-c69f2eea52e3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.613528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c20e0b-e8fd-428e-9630-c69f2eea52e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.636188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-24c20e0b-e8fd-428e-9630-c69f2eea52e3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.637431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-24c20e0b-e8fd-428e-9630-c69f2eea52e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.643165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 792/1563] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.646275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 772/1564] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.647644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e19fcf-aaa3-478f-b8bc-9eebb6c3a1d2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.647943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e19fcf-aaa3-478f-b8bc-9eebb6c3a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.648230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e19fcf-aaa3-478f-b8bc-9eebb6c3a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.650890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.662125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c212f-4366-48fe-983b-cd294c524f2e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.666720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.667242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a4e19fcf-aaa3-478f-b8bc-9eebb6c3a1d2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.667717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.667717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.667929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.668444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a4e19fcf-aaa3-478f-b8bc-9eebb6c3a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.668648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:59.668962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.671415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ba339-57d5-4b78-a4dd-7f7dbd9d94f6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.677044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 773/1565] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.678311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb5152-0f47-4704-8b06-7cb8bf62ea47 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.678610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb5152-0f47-4704-8b06-7cb8bf62ea47 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.678897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb5152-0f47-4704-8b06-7cb8bf62ea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.683689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 793/1566] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:59.684940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.685839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.686099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.703898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-38bb5152-0f47-4704-8b06-7cb8bf62ea47 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.704100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.705039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-38bb5152-0f47-4704-8b06-7cb8bf62ea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.705238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.713327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 774/1567] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.713472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 794/1568] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.714027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.718862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.719164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.719450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.727588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4494a-022e-4a9e-87d3-6d7e31d96f52 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.732272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.733226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.733616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.733951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.734699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:59.735013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.736625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82960f73-03b5-4574-b761-b0d364d853ca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.737892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.739106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.747118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 775/1569] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.748114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.755399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 795/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:59.755399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.755399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.755399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.761868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a567b3-235f-4768-a123-1d8c1ae91db6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.768566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.768881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.769256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.769565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.769920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.770131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.778151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.778631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 796/1571] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.780076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.780645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.789476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d62a45-1ec3-40ef-b8af-ece377d85aab None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.790017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 776/1572] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:42:59.790800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.794162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.795529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.796001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.796290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.797026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:59.797341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.798980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.801172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986eeee-d1a5-4c4e-8b9c-b0b561f11f31 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.805761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bab416-038e-4369-a78c-35e9ee9e157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.806410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bab416-038e-4369-a78c-35e9ee9e157a admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.806703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bab416-038e-4369-a78c-35e9ee9e157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.806993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bab416-038e-4369-a78c-35e9ee9e157a admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.807686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bab416-038e-4369-a78c-35e9ee9e157a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.809554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bab416-038e-4369-a78c-35e9ee9e157a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.811296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 797/1573] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:59.811947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.822075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0e40-4bf1-4716-8a16-c1191df94d5d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.828507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.829541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.830022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.830477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.831488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:59.832066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.833912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.845192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 777/1574] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:59.848804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869b76-e224-489b-a098-4d271146c96c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.849095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869b76-e224-489b-a098-4d271146c96c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.849388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869b76-e224-489b-a098-4d271146c96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.850458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 798/1575] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:59.855192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.868635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e549-d44f-4c75-919b-9c00dcd605dc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.868836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b0869b76-e224-489b-a098-4d271146c96c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:42:59.870294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b0869b76-e224-489b-a098-4d271146c96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:42:59.875672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:59.931953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.932625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.932920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.933203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.933996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:59.934283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.935925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.936195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 778/1576] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:42:59.938521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0869b76-e224-489b-a098-4d271146c96c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.948081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0869b76-e224-489b-a098-4d271146c96c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.952376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 799/1577] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:59.953216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.953507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.954447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.954927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.955410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.956292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.958258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:42:59.958643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.964939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d6a40-b678-48fb-b3d0-43aed817a888 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:42:59.970469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:42:59.971486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:42:59.971977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:42:59.972455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:42:59.973379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:42:59.973938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:42:59.975610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:42:59.990114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 800/1578] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:42:59.991030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:42:59.993095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 779/1579] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/projects => generated 361 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:42:59.994193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.006646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e610-c2fd-48dc-b9fb-973a7132e08a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.062400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748106f-9563-4fdf-8caf-3035e0953662 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.067084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.067815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.068049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.068334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.069032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.069557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.070071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:43:00.070071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:43:00.070071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:43:00.070371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.070593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.070879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.071428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.071720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.072755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:43:00.072755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:43:00.072755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:43:00.074016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.094828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 780/1580] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.095867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.103088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 801/1581] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:42:59 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.104176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.108708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9075e2-db99-4f18-b387-ecf3a85c2036 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.164774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.165452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.165779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.166067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.166639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d38f5-c8a1-46e3-81e5-c3fa1bf77778 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.166763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.168699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.171380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.172191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.172390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.172732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.173519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:00.173879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.175475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.188223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 802/1582] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:00.189050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.197600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 781/1583] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.198405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.203250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.203569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8804dd3-b705-4b82-84cc-c25bac7bc427 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.260701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572378b-0555-4a94-9f2c-9aae93499604 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.261725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb41f5-e236-43b4-acf2-3f76fd40c965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.262406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb41f5-e236-43b4-acf2-3f76fd40c965 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.262701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb41f5-e236-43b4-acf2-3f76fd40c965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.263069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb41f5-e236-43b4-acf2-3f76fd40c965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.264275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb41f5-e236-43b4-acf2-3f76fd40c965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.266199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.266343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb41f5-e236-43b4-acf2-3f76fd40c965 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.266875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.267164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.267452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.268141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.270091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.281064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.295082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 782/1584] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.295082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.300528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 803/1585] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.300528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc76a7a-33ca-4e6c-9fba-48bf4a696d66 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.300528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc76a7a-33ca-4e6c-9fba-48bf4a696d66 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.300528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc76a7a-33ca-4e6c-9fba-48bf4a696d66 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.301654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 804/1586] 10.222.0.57 () {54 vars in 912 bytes} [Wed Jun 15 14:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:00.302229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc76a7a-33ca-4e6c-9fba-48bf4a696d66 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.303243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a014c-cc0a-42cf-ab42-dc00537336e0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.362066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.362744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.363036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.363331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.364100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.366058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.367796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc76a7a-33ca-4e6c-9fba-48bf4a696d66 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.372790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.374132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.374423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.374697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.375915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.377555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.388968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 805/1587] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.390056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.395856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 783/1588] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.396802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.400481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ab34-8f47-4c82-a616-3be0062f4e6e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.460026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7f3ff-3f20-4912-bf76-83c4638172e1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.464789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.465345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.465483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.465756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.466041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.466145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.467002 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.467321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.467498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.468101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.469222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.470075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.490298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 806/1589] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.490298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.498901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 784/1590] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.499975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.501019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593084-ba10-4f75-aa75-e2f6fa3e518a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.504213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.559729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b6b13-b17a-4eb3-84ea-0d215cf34030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.561010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6b13-b17a-4eb3-84ea-0d215cf34030 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.561291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6b13-b17a-4eb3-84ea-0d215cf34030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.561575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6b13-b17a-4eb3-84ea-0d215cf34030 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.562304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b6b13-b17a-4eb3-84ea-0d215cf34030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.564240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b6b13-b17a-4eb3-84ea-0d215cf34030 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.565149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fc324-6379-4944-a133-cd5746df1e99 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.569840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.570541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.570840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.571124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.571866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.573458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.583988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 785/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.584926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 807/1592] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.586099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.586397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.586701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.587900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.597622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07a096-653d-4a4b-b54c-a8f62340be10 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.602369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.603032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.603322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.603610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.604307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.604785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:00.606038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:00.606255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.610968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.630313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 786/1593] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.631151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.640376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a602f6-8a68-4ff9-8e9e-c9e6b568d906 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.645100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.645796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.646090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.646380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.647108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.648699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.660062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 787/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.660860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.670891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa29b83-7189-45c0-9d88-3fc64d48f020 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.675763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.676890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.677352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.677686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.678396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.680305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.686348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 808/1595] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:00.687286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.699606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d969b9-e0f0-488b-91d4-c27529fd37a0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.701539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 788/1596] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.702310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.705473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.706207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.706498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.706779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.707766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.709698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.712482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e7f8d-7e0e-43ce-9283-3b11f3737dab admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.718197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.718861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.719287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.719579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.720274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.720565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 809/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.723657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.723741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.734062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c57eee-1457-450c-b8e4-a46565bbbd55 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.739470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.740174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.740469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.740748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.741482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.743116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.746384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 789/1598] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:00 2022] POST /identity/v3/users => generated 302 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:00.746384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.754483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679c81e-002e-4c3a-994e-6466a25e4f42 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.756637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 810/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.757456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.762318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.762985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.763936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.764231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.765060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.766694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.769817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cb639-dbbc-45a6-b9ce-dea911bc4b17 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.774384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.775036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.775327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.775609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.776337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.777193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 790/1600] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.778136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.781553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.788508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6cffb-38eb-4291-b125-310e6dc1f782 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.796471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 811/1601] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.796637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.796769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.796861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.796861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.796861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] PATH_INFO: `/v3/projects/4bf1cdbd5dc9471f9f1732600ab98c24/users/1d58ac7f49454fd6a5b5eef6a1e8b2d8/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.810338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf1cdbd5dc9471f9f1732600ab98c24, user_id=1d58ac7f49454fd6a5b5eef6a1e8b2d8, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.812994 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.827178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 791/1602] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:00 2022] PUT /identity/v3/projects/4bf1cdbd5dc9471f9f1732600ab98c24/users/1d58ac7f49454fd6a5b5eef6a1e8b2d8/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:00.828000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.851362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc117d-0c30-409c-865e-d4ff571b2b80 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.892314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10790f6-31d7-4276-8a46-aee6733b0c17 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.897230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.897935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.898230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.898528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.899278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.900363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.900912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.901018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.901294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.901577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.902329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.913494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 792/1603] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.914435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.923994 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77670c5a-77c0-4460-a1c0-554a79698612 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.928686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.929336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.929628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.929952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.930673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.932264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.942796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 793/1604] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.943027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:00.943684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.954224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa060ecc-501e-4d98-93f3-7f46074246d2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.958858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.959759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.960054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.960335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.961053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.962658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.973102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 794/1605] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:00.974545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.974947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:00.976399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:00.984997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70548e8-ac3b-4c79-b8cc-7a9ef7bd955d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:00.987532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 812/1606] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:00.988166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:00.990812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db00d78-d1ff-42ae-b70b-a84e29a3f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:00.991696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00d78-d1ff-42ae-b70b-a84e29a3f168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:00.991989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00d78-d1ff-42ae-b70b-a84e29a3f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:00.992273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00d78-d1ff-42ae-b70b-a84e29a3f168 admin admin] PATH_INFO: `/v3/projects/177bfdaa8f9f4b6b944dc2e9362abec4/users/19f98fc672014da1a8facd4d4d7495bb/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:00.998537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab1938-1d73-4235-9a34-cbfb034b0701 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.004476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.005807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.006089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.006370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.007120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.008594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db00d78-d1ff-42ae-b70b-a84e29a3f168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=177bfdaa8f9f4b6b944dc2e9362abec4, user_id=19f98fc672014da1a8facd4d4d7495bb, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.008731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.011717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db00d78-d1ff-42ae-b70b-a84e29a3f168 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.020524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 813/1607] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:01.022166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.037860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0986d0e-6e41-4358-8545-d03e76beaa16 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.038095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 795/1608] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:00 2022] PUT /identity/v3/projects/177bfdaa8f9f4b6b944dc2e9362abec4/users/19f98fc672014da1a8facd4d4d7495bb/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:01.039215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.039518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.039830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.059655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.061905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:01.103061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.104074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.104551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.105006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.106057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.108438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.120299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 814/1609] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:01.121402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.130437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 796/1610] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:01.131254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.135811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e467d4-a735-42e1-88a5-117a038fb0d4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.141081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.141616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9675f-246f-4c5c-b54e-3ff5f7f587c9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.141877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.142176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.142464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.143207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.144816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.150997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e89124-ad0e-4d6f-a35c-e59e911f92d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.151917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89124-ad0e-4d6f-a35c-e59e911f92d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.152491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89124-ad0e-4d6f-a35c-e59e911f92d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.152784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89124-ad0e-4d6f-a35c-e59e911f92d2 admin admin] PATH_INFO: `/v3/projects/26d36b3b04f046d3bee421c5987b965b/users/ed7372fed3e14f3e98ca25d22a7c7a7e/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.155770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 815/1611] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:01.156433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.166233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4965720-9f8f-4512-800f-f4259cee76f2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.171619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.172112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e89124-ad0e-4d6f-a35c-e59e911f92d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d36b3b04f046d3bee421c5987b965b, user_id=ed7372fed3e14f3e98ca25d22a7c7a7e, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.172313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.172577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.173316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.173594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.174868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e89124-ad0e-4d6f-a35c-e59e911f92d2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.175336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.189917 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 816/1612] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:01.190745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.194915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 797/1613] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:01 2022] PUT /identity/v3/projects/26d36b3b04f046d3bee421c5987b965b/users/ed7372fed3e14f3e98ca25d22a7c7a7e/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:01.196107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.196405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.196692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.203343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b274ca-b015-41b2-a489-3b837b2dc213 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.233059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] MFA Rules not processed for user `19f98fc672014da1a8facd4d4d7495bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.236038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:01.271793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.272874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.273173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.273520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] PATH_INFO: `/v3/projects/c6d8b04fcde14efe8dae7d53b149196c/users/c8bfa2079dd64720968479ee35176fa2/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.295589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d8b04fcde14efe8dae7d53b149196c, user_id=c8bfa2079dd64720968479ee35176fa2, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.296793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:43:01.296793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:43:01.296793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:43:01.299310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.315184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 817/1614] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:01 2022] PUT /identity/v3/projects/c6d8b04fcde14efe8dae7d53b149196c/users/c8bfa2079dd64720968479ee35176fa2/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:01.315970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.325061 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 798/1615] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:01.326627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.332575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:01.344487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:01.387036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec133c-7ad6-4ea2-b45a-3a999310015a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.393055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c104db9-33f6-47aa-8d39-1c9796631e8d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.398667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.399801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.400107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.400365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] PATH_INFO: `/v3/projects/0c12c8e1398843eca8746543a0684841/users/28ff37865a9b49ec8601140fca00a627/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.417367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c12c8e1398843eca8746543a0684841, user_id=28ff37865a9b49ec8601140fca00a627, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.420874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.439480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 818/1616] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:01 2022] PUT /identity/v3/projects/0c12c8e1398843eca8746543a0684841/users/28ff37865a9b49ec8601140fca00a627/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:01.440506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.448943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.450052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.450501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.451530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.468587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.470343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.483083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 799/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:01.483903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.493849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de02f-d729-4ff3-967c-f83b71581d17 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.561610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92096d1-6ac2-4289-91da-be662d4e2718 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.572730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fbd8-2ccc-4869-80a3-35913caa69b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.573751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fbd8-2ccc-4869-80a3-35913caa69b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.577201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fbd8-2ccc-4869-80a3-35913caa69b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.577638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fbd8-2ccc-4869-80a3-35913caa69b4 admin admin] PATH_INFO: `/v3/projects/3f4caa81d58a4daf90980e624d1c2526/users/05add5effa924b908fc522cb16f8387c/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.597564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b022ec-ea7d-42ca-8ed1-bd7150c145bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.597564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b022ec-ea7d-42ca-8ed1-bd7150c145bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.597564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b022ec-ea7d-42ca-8ed1-bd7150c145bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.598682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b022ec-ea7d-42ca-8ed1-bd7150c145bd admin admin] PATH_INFO: `/v3/projects/459c99f4d26041ccbc68b1a0c660570a/users/91bc239ccbb047578e896bd984ea420b/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.617626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fbd8-2ccc-4869-80a3-35913caa69b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f4caa81d58a4daf90980e624d1c2526, user_id=05add5effa924b908fc522cb16f8387c, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.624049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fbd8-2ccc-4869-80a3-35913caa69b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.635442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b022ec-ea7d-42ca-8ed1-bd7150c145bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=459c99f4d26041ccbc68b1a0c660570a, user_id=91bc239ccbb047578e896bd984ea420b, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.639554 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b022ec-ea7d-42ca-8ed1-bd7150c145bd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.651892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 819/1618] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:01 2022] PUT /identity/v3/projects/3f4caa81d58a4daf90980e624d1c2526/users/05add5effa924b908fc522cb16f8387c/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:01.653503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.654409 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.654710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.679956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 800/1619] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:01 2022] PUT /identity/v3/projects/459c99f4d26041ccbc68b1a0c660570a/users/91bc239ccbb047578e896bd984ea420b/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:01.681430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f69335-537a-49c4-9fd3-3fd9fb8bdabd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.682786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f69335-537a-49c4-9fd3-3fd9fb8bdabd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.683362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f69335-537a-49c4-9fd3-3fd9fb8bdabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.693891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.695191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:01.736670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-38f69335-537a-49c4-9fd3-3fd9fb8bdabd None None] MFA Rules not processed for user `c8bfa2079dd64720968479ee35176fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.738054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-38f69335-537a-49c4-9fd3-3fd9fb8bdabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:01.797459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:01.813815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 820/1620] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:01.814742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:01.833851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 801/1621] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:01.835756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92f796-437e-42d8-95d0-8472e4e9faa4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.836264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92f796-437e-42d8-95d0-8472e4e9faa4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.836694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92f796-437e-42d8-95d0-8472e4e9faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.838459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef29e6e-5aab-43ac-8877-57375957faed None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:01.844281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6843514-3e47-45c4-8356-374aeb11fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:01.844973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6843514-3e47-45c4-8356-374aeb11fc1c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.845267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6843514-3e47-45c4-8356-374aeb11fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.845604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6843514-3e47-45c4-8356-374aeb11fc1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.846358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6843514-3e47-45c4-8356-374aeb11fc1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:01.848307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6843514-3e47-45c4-8356-374aeb11fc1c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:01.865089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3d92f796-437e-42d8-95d0-8472e4e9faa4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.867543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:01.869502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3d92f796-437e-42d8-95d0-8472e4e9faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:01.885530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 802/1622] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:01.885530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.885530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.885530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.892741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 821/1623] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:01.897020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:01.898328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:01.898628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:01.918609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.919939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:01.936207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:01.937453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:02.003223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 822/1624] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:02.004398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.015887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 803/1625] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:02.017012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.065813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fa78d-0250-4235-a11a-4d34e071a1cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.070466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.071140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.071432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.071719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.082793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.084317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.095577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 823/1626] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:02.096253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.105537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45a07c-c0f5-4540-9968-4085fe5e65f6 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.110258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.110935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.111228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.111517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.112223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.114178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.119432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da3c3e-6fa0-471d-9aad-17c9c0c62d9a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.124755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.125726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.126091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.126596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.136436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 824/1627] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:02 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:02.137395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.147318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4785ab-55c6-4ab6-844b-e77ceaaa1146 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.151208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4a06-471f-4d85-9076-b81376d989d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.152299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4a06-471f-4d85-9076-b81376d989d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.152299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4a06-471f-4d85-9076-b81376d989d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.152694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4a06-471f-4d85-9076-b81376d989d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.165200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4a06-471f-4d85-9076-b81376d989d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.166791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4a06-471f-4d85-9076-b81376d989d5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.177928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 825/1628] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:02.179588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cb710-4cb8-40fa-8d89-149924141bb3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.179915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cb710-4cb8-40fa-8d89-149924141bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.180500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cb710-4cb8-40fa-8d89-149924141bb3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.182347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 826/1629] 10.222.0.57 () {54 vars in 961 bytes} [Wed Jun 15 14:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:02.183282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cb710-4cb8-40fa-8d89-149924141bb3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.196536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cb710-4cb8-40fa-8d89-149924141bb3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.200760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.201444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.201874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.202283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.214036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.215087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.215982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.217761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.231120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 827/1630] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:02.232610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.239562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 804/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:02.240244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.243111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed602085-30ce-4094-92b7-f7844c0e9b32 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.248744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9f7a9-d237-469d-8517-a972b8481961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.249512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9f7a9-d237-469d-8517-a972b8481961 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.249965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9f7a9-d237-469d-8517-a972b8481961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.250346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9f7a9-d237-469d-8517-a972b8481961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.254799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c67b-9d48-4a86-8fbd-b10dd6b8f750 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.262098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.263980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9f7a9-d237-469d-8517-a972b8481961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.265247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.265971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.266625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9f7a9-d237-469d-8517-a972b8481961 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.267118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.268742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.270471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.296701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 828/1632] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:02.298015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd88e2e-132d-441a-9b3f-9ecd65fda983 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.298754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd88e2e-132d-441a-9b3f-9ecd65fda983 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.298879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd88e2e-132d-441a-9b3f-9ecd65fda983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.304159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 805/1633] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:02.308181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.334870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d0411-8158-48e8-8fa6-e3315a7b5d92 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.339604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ecd88e2e-132d-441a-9b3f-9ecd65fda983 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:02.341856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ecd88e2e-132d-441a-9b3f-9ecd65fda983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:02.347509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.351124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.351124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.351361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] PATH_INFO: `/v3/projects/aeac138c77ef486b81ae7ee289669f84/users/7e759c048c1a42309f408eea0bee7c18/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.392417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeac138c77ef486b81ae7ee289669f84, user_id=7e759c048c1a42309f408eea0bee7c18, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.399612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.405431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:02.416024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:02.427137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 806/1634] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:02 2022] PUT /identity/v3/projects/aeac138c77ef486b81ae7ee289669f84/users/7e759c048c1a42309f408eea0bee7c18/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:02.433773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.480642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 829/1635] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Jun 15 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:02.490900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67effcb3-3036-4362-ade3-9c929833d548 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.491508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67effcb3-3036-4362-ade3-9c929833d548 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.492084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67effcb3-3036-4362-ade3-9c929833d548 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.494973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 830/1636] 10.222.0.57 () {54 vars in 912 bytes} [Wed Jun 15 14:43:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:02.504592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.505551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.505909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.544219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:02.549453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:02.561757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe561b-31ca-49f0-a64b-3bca50d568ed admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.575237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.578643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.579676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.579676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.581783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.587179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.618493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 807/1637] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:02.619537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.650118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cf5d3-9368-409f-959c-bbe52965837d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.675685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 831/1638] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:02.677194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8582f-f4f0-4d81-9258-07300f46219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.678421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8582f-f4f0-4d81-9258-07300f46219b admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.679046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8582f-f4f0-4d81-9258-07300f46219b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.679331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8582f-f4f0-4d81-9258-07300f46219b admin admin] PATH_INFO: `/v3/projects/aeac138c77ef486b81ae7ee289669f84/users/7e759c048c1a42309f408eea0bee7c18/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.683207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8582f-f4f0-4d81-9258-07300f46219b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeac138c77ef486b81ae7ee289669f84, user_id=7e759c048c1a42309f408eea0bee7c18, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:02.687009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8582f-f4f0-4d81-9258-07300f46219b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:02.698725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:02.728281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 808/1639] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:02 2022] PUT /identity/v3/projects/aeac138c77ef486b81ae7ee289669f84/users/7e759c048c1a42309f408eea0bee7c18/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:02.743348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.752745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.753211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.823698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] MFA Rules not processed for user `7e759c048c1a42309f408eea0bee7c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:02.824958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:02.843067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd338ad6-3dbf-4db5-ac01-aec6fa02dfcd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:02.912363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:02.913045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:02.913337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:02.913730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:02.977486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:03.012133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 809/1640] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:03.062166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:03.075042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4d205-7c8f-4e1e-b2c3-d9c9e658d3ed None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:03.082832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:03.084846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:03.087262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69951b1d-830e-4aac-87f9-7d496f6b61b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:03.089643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69951b1d-830e-4aac-87f9-7d496f6b61b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:03.089643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69951b1d-830e-4aac-87f9-7d496f6b61b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:03.089643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69951b1d-830e-4aac-87f9-7d496f6b61b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:03.101188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 832/1641] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:03.108386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69951b1d-830e-4aac-87f9-7d496f6b61b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:03.110232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69951b1d-830e-4aac-87f9-7d496f6b61b8 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:03.140643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 810/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:03.294744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:03.308236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7c1b-28c8-4c89-94ff-a0ffeb19122c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:03.315916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f0441-ee5b-4056-b14f-7bf47b896fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:03.319249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f0441-ee5b-4056-b14f-7bf47b896fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:03.319543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f0441-ee5b-4056-b14f-7bf47b896fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:03.319827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f0441-ee5b-4056-b14f-7bf47b896fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:03.472375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f0441-ee5b-4056-b14f-7bf47b896fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:03.474055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f0441-ee5b-4056-b14f-7bf47b896fa3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:03.484662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 833/1643] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:04.771957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:04.772787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:04.772787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:04.831406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] MFA Rules not processed for user `05add5effa924b908fc522cb16f8387c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:04.832662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:04.954950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:05.010045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 811/1644] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:05.016682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7c7cc-9b3a-4bbf-ab73-562b5caa0fff None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:05.017248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7c7cc-9b3a-4bbf-ab73-562b5caa0fff None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:05.017248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7c7cc-9b3a-4bbf-ab73-562b5caa0fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:05.049763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0ef7c7cc-9b3a-4bbf-ab73-562b5caa0fff None None] MFA Rules not processed for user `05add5effa924b908fc522cb16f8387c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:05.051008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0ef7c7cc-9b3a-4bbf-ab73-562b5caa0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:05.068736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 834/1645] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:05.080844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:05.105722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c61bcc-acc8-4abf-88f1-b4278bb8516b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:05.114303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:05.115282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:05.115741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:05.116501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:05.155402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:05.157496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:05.179643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 812/1646] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:05.880451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:05.880982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:05.881298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:05.928425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] MFA Rules not processed for user `91bc239ccbb047578e896bd984ea420b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:05.929768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:06.086944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.089161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 835/1647] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:06.253326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec61ee-db91-47a8-b6a9-ad04e5f10b08 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:06.273318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83616340-cd9d-4f7b-99b1-a27cd5d5aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:06.274254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83616340-cd9d-4f7b-99b1-a27cd5d5aaaa admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.274812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83616340-cd9d-4f7b-99b1-a27cd5d5aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.275350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83616340-cd9d-4f7b-99b1-a27cd5d5aaaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.276455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83616340-cd9d-4f7b-99b1-a27cd5d5aaaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.278658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83616340-cd9d-4f7b-99b1-a27cd5d5aaaa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.348822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 813/1648] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:06 2022] POST /identity/v3/projects => generated 403 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:06.353251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.376517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c12b-ac00-415f-a44b-59f2603180bb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:06.380131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a223b-730e-42dd-8e3c-dcedaf094532 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.380683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a223b-730e-42dd-8e3c-dcedaf094532 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.381129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a223b-730e-42dd-8e3c-dcedaf094532 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.384362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 814/1649] 10.222.0.57 () {54 vars in 912 bytes} [Wed Jun 15 14:43:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:06.391045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.391817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.392270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.508472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:06.559130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] MFA Rules not processed for user `d292189b741b40b8859bfe9883b4c5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:06.559130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:06.613180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:06.614001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.619863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.620518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.621430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.641773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.694224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 836/1650] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:06 2022] POST /identity/v3/users => generated 308 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:06.702833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.706304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 815/1651] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:43:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:06.723405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c481-52e7-4630-8c32-0376923736f1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:06.746838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:06.748059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.748351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.748653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.749516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.751300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.761513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.768845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 837/1652] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:06.769821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.777956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bfda7-5ae1-4116-ac11-d0c9f40c0228 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:06.790870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:06.796881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6542e8-9fc9-45c4-b42b-c86263db86f2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:06.798031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.798463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.798957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.809609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:06.811173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:06.811462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:06.811743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] PATH_INFO: `/v3/projects/f0ea6519831d4e559f12808f0050709e/users/76bbde0223de4b1883b288fdf9df8225/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:06.817085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.820254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.847853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 816/1653] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:06.855360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ea6519831d4e559f12808f0050709e, user_id=76bbde0223de4b1883b288fdf9df8225, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:06.859400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:06.897447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 838/1654] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:06 2022] PUT /identity/v3/projects/f0ea6519831d4e559f12808f0050709e/users/76bbde0223de4b1883b288fdf9df8225/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:06.901462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:06.935834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcbbad-7550-4ebc-bc73-28f90150e289 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:07.013049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:07.049433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:07.050348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:07.050909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:07.051355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:07.052382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:07.077712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:07.115743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:07.116387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 817/1655] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:07.129371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fd87a-009a-4f23-bac0-4bde3f349dc7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:07.146752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881b6bd-fc2d-4ef4-84a4-5bbc0f6bb95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:07.148017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881b6bd-fc2d-4ef4-84a4-5bbc0f6bb95b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:07.150245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881b6bd-fc2d-4ef4-84a4-5bbc0f6bb95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:07.151499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881b6bd-fc2d-4ef4-84a4-5bbc0f6bb95b admin admin] PATH_INFO: `/v3/projects/f0ea6519831d4e559f12808f0050709e/users/76bbde0223de4b1883b288fdf9df8225/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:07.155298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881b6bd-fc2d-4ef4-84a4-5bbc0f6bb95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ea6519831d4e559f12808f0050709e, user_id=76bbde0223de4b1883b288fdf9df8225, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:07.159305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881b6bd-fc2d-4ef4-84a4-5bbc0f6bb95b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:07.180824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 839/1656] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:07 2022] PUT /identity/v3/projects/f0ea6519831d4e559f12808f0050709e/users/76bbde0223de4b1883b288fdf9df8225/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:08.413897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:08.430800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011e86-cc46-4d95-9e3d-2d5248f0e01e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:08.741380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:08.742912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:08.743496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:08.744041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] PATH_INFO: `/v3/users/19f98fc672014da1a8facd4d4d7495bb` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:08.745695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f98fc672014da1a8facd4d4d7495bb)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:08.749749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:08.761903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1393-0bb0-4f88-ba2a-e88fae78e79e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:08.763779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1393-0bb0-4f88-ba2a-e88fae78e79e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:08.764650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1393-0bb0-4f88-ba2a-e88fae78e79e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:08.766535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 840/1657] 10.222.0.57 () {54 vars in 960 bytes} [Wed Jun 15 14:43:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:08.773308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:08.773925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:08.776706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:08.851754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:08.854938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:08.908261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f98fc672014da1a8facd4d4d7495bb'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:08.944218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f98fc672014da1a8facd4d4d7495bb'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:08.984443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f98fc672014da1a8facd4d4d7495bb'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:08.998639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f98fc672014da1a8facd4d4d7495bb'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:09.001555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-448861b7-44c2-4564-8a56-6ec6b8d71596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f98fc672014da1a8facd4d4d7495bb'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:09.024740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 818/1658] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:08 2022] DELETE /identity/v3/users/19f98fc672014da1a8facd4d4d7495bb => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:09.029769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:09.030292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:09.030819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:09.046357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 841/1659] 10.222.0.57 () {58 vars in 1057 bytes} [Wed Jun 15 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:09.055519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:09.080612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ab160-94b3-484c-95d5-87d8541f0e15 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:09.100953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:09.115293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:09.235689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:09.237853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:09.238205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:09.238538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:09.243256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:09.272712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:09.274428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:09.276718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:09.299632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 819/1660] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:09.300286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 842/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:09.355590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:09.358223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:09.561814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba60af-c09e-4ced-a2d2-efaecc477a2d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:09.570204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8afe5-c17a-48a5-9ca0-53ec0c0f5f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:09.571653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8afe5-c17a-48a5-9ca0-53ec0c0f5f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:09.572104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8afe5-c17a-48a5-9ca0-53ec0c0f5f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:09.572543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8afe5-c17a-48a5-9ca0-53ec0c0f5f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:09.593223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8afe5-c17a-48a5-9ca0-53ec0c0f5f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:09.595148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8afe5-c17a-48a5-9ca0-53ec0c0f5f3b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:09.610891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 820/1662] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:11.365709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:11.367820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:11.406894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8756d0-af21-41a6-b813-af851c413b07 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:11.420926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:11.422368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:11.422773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:11.423163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] PATH_INFO: `/v3/users/4735488a206c4a2da7873c310a95a91e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:11.425687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4735488a206c4a2da7873c310a95a91e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:11.426929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:43:11.426929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Jun 15 14:43:11.426929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: Jun 15 14:43:11.430688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:11.462584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:11.604220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4735488a206c4a2da7873c310a95a91e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.608901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4735488a206c4a2da7873c310a95a91e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.666721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4735488a206c4a2da7873c310a95a91e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.672173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4735488a206c4a2da7873c310a95a91e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.688458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73c6d720-a735-4f34-89cb-bf3550629314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4735488a206c4a2da7873c310a95a91e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:11.798971 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 843/1663] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:11 2022] DELETE /identity/v3/users/4735488a206c4a2da7873c310a95a91e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:11.801221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:11.802391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:11.802391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:11.843618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:11.845467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:11.876502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:11.877362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:11.877921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:11.972214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] MFA Rules not processed for user `76bbde0223de4b1883b288fdf9df8225`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:11.977929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:12.104171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 821/1664] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:12.105197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:12.109799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:12.144797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dce2d8-88a5-4c1c-b591-714b1cb13784 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:12.161220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:12.162414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:12.164091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:12.164495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] PATH_INFO: `/v3/projects/177bfdaa8f9f4b6b944dc2e9362abec4` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:12.168540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=177bfdaa8f9f4b6b944dc2e9362abec4)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:12.171577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:43:12.171577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:43:12.171577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:43:12.174602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:12.189626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 844/1665] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:12.190705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:12.193142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:12.281100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cafbae-f897-4563-aed0-4c23340b8f67 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:12.290776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221e33d-6935-457a-b34b-f060915bacf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:12.291976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221e33d-6935-457a-b34b-f060915bacf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:12.292613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221e33d-6935-457a-b34b-f060915bacf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:12.293360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221e33d-6935-457a-b34b-f060915bacf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:12.335280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221e33d-6935-457a-b34b-f060915bacf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:12.337144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221e33d-6935-457a-b34b-f060915bacf6 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:12.353793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 845/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:12.365342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177bfdaa8f9f4b6b944dc2e9362abec4', 'request_id': 'req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.384678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177bfdaa8f9f4b6b944dc2e9362abec4', 'request_id': 'req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.418837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177bfdaa8f9f4b6b944dc2e9362abec4', 'request_id': 'req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.451552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] The token cache is being invalidate because project 177bfdaa8f9f4b6b944dc2e9362abec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:12.452171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.464648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.470501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:12.471230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 822/1667] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:12 2022] DELETE /identity/v3/projects/177bfdaa8f9f4b6b944dc2e9362abec4 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:12.471544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:12.471962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:12.472350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:12.474354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:12.509164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:12.527372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] MFA Rules not processed for user `1d58ac7f49454fd6a5b5eef6a1e8b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:12.529490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:12.543390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:12.683831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2d991-5c3d-4a56-8dc2-c17fa30b8078 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:12.696078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:12.698381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:12.698683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:12.698971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] PATH_INFO: `/v3/users/7e759c048c1a42309f408eea0bee7c18` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:12.702805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e759c048c1a42309f408eea0bee7c18)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:12.705099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 846/1668] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:12.707116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:12.714095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:12.715819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:12.820713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e759c048c1a42309f408eea0bee7c18'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.862830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e759c048c1a42309f408eea0bee7c18'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.874557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72cf81-89cc-471f-ac43-76f15a825498 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:12.879009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e759c048c1a42309f408eea0bee7c18'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.890926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:12.891745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:12.891902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:12.892170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:12.897429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e759c048c1a42309f408eea0bee7c18'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.898297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-14914857-4699-4d81-b44f-56923f4b07ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e759c048c1a42309f408eea0bee7c18'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:12.913625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:12.915290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:12.916275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 823/1669] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:12 2022] DELETE /identity/v3/users/7e759c048c1a42309f408eea0bee7c18 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:12.932821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 847/1670] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:13.113182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:13.113485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:13.113857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:13.151128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:13.154799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:13.374411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 824/1671] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:13.727436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:13.729596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:13.876924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b6a46-d914-42af-95d0-0871d3504ec7 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:13.912025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:13.987129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d82b1f-4413-4419-b060-1a60128095d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:13.987906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82b1f-4413-4419-b060-1a60128095d0 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:13.988271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82b1f-4413-4419-b060-1a60128095d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:13.988625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82b1f-4413-4419-b060-1a60128095d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:14.014146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d82b1f-4413-4419-b060-1a60128095d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:14.016099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d82b1f-4413-4419-b060-1a60128095d0 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:14.035764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 848/1672] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:43:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:43:14.335327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:14.337726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:14.482163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:14.527263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024da2-ca3e-4440-a910-9d97e46b1468 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:14.543486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:14.550499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:14.551277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:14.551783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] PATH_INFO: `/v3/projects/aeac138c77ef486b81ae7ee289669f84` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:14.558493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeac138c77ef486b81ae7ee289669f84)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:14.567535 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:14.700857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeac138c77ef486b81ae7ee289669f84', 'request_id': 'req-7b83a262-efa7-49bd-b649-ddf579085fb1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.719755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeac138c77ef486b81ae7ee289669f84', 'request_id': 'req-7b83a262-efa7-49bd-b649-ddf579085fb1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.732485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeac138c77ef486b81ae7ee289669f84', 'request_id': 'req-7b83a262-efa7-49bd-b649-ddf579085fb1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.800623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] The token cache is being invalidate because project aeac138c77ef486b81ae7ee289669f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:14.802151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.804050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b83a262-efa7-49bd-b649-ddf579085fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:14.814408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 825/1673] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:14 2022] DELETE /identity/v3/projects/aeac138c77ef486b81ae7ee289669f84 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:14.816667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:14.817184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:14.817268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:14.880769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:14.883836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.082532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 849/1674] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.084262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.085377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.085691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.124723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:15.126086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.144160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 826/1675] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:15.146054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.165276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcc82-0b10-4b14-95aa-41b08c6835a1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.180551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.182671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.182991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.183596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.184011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:15.184330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.195363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.237552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 850/1676] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:15.238128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.269881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796bdd-4289-47cb-b5dc-3e7c765adf51 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.292022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.295761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.297044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.297575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.299869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.312175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.402444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 827/1677] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/projects => generated 407 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:15.406398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.424215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12983c7-93f7-4fe9-9825-046cc338d43e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.600894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.603019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.603327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.603612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.604337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.610545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.657145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.659325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.681638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 851/1678] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:15 2022] POST /identity/v3/users => generated 304 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:15.684596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.689689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35cf35-73a6-463a-9403-3409c2a25d48 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.703073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.707195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.708224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.709400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] PATH_INFO: `/v3/users/c8bfa2079dd64720968479ee35176fa2` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.712877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8bfa2079dd64720968479ee35176fa2)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.718701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd70ee-8efe-4783-88cb-b70ef9c713f5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.722818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.737824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.738644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.739008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.740474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.741255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.742936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.810642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 852/1679] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:15.814732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:15.823778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:15.895665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1bf03-b624-44e3-98da-f72e79c5211f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:15.924392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8bfa2079dd64720968479ee35176fa2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.924972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:15.926812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:15.927110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:15.927398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] PATH_INFO: `/v3/projects/658ef56d424e4381a1fba60ebedf0277` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:15.929155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658ef56d424e4381a1fba60ebedf0277)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:15.932083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:15.977882 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8bfa2079dd64720968479ee35176fa2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:15.986228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:15.994015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8bfa2079dd64720968479ee35176fa2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.008538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8bfa2079dd64720968479ee35176fa2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.018794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8bfa2079dd64720968479ee35176fa2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.045759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 828/1680] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:15 2022] DELETE /identity/v3/users/c8bfa2079dd64720968479ee35176fa2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:16.047990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.049858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.056646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658ef56d424e4381a1fba60ebedf0277', 'request_id': 'req-049913c5-e40c-4762-a013-ea4fba35fc58'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.074978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658ef56d424e4381a1fba60ebedf0277', 'request_id': 'req-049913c5-e40c-4762-a013-ea4fba35fc58'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.089570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658ef56d424e4381a1fba60ebedf0277', 'request_id': 'req-049913c5-e40c-4762-a013-ea4fba35fc58'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.112301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:16.133270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] The token cache is being invalidate because project 658ef56d424e4381a1fba60ebedf0277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:16.136527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.140083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-049913c5-e40c-4762-a013-ea4fba35fc58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:16.151602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 853/1681] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:15 2022] DELETE /identity/v3/projects/658ef56d424e4381a1fba60ebedf0277 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:16.153369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.162174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.177011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.274018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:16.282334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.332043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0989cc0-97f4-4c2e-bb8d-6a1aba65891b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.348107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03386-d7f9-4b67-a3c4-7e879b684959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.349969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03386-d7f9-4b67-a3c4-7e879b684959 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.350781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03386-d7f9-4b67-a3c4-7e879b684959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.351290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03386-d7f9-4b67-a3c4-7e879b684959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.352293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03386-d7f9-4b67-a3c4-7e879b684959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.360440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03386-d7f9-4b67-a3c4-7e879b684959 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.397552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 829/1682] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:16.401078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.402066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.403251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.430654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 854/1683] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:16.432197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.438751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.465596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:16.487951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.493841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e04a23-0d9d-4ca7-b0d4-d78641c4068f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.516247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e25ac6-0a1c-4d16-8efa-90b6b2c6b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.518469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 830/1684] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:16.519388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25ac6-0a1c-4d16-8efa-90b6b2c6b7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.519775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25ac6-0a1c-4d16-8efa-90b6b2c6b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.521644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25ac6-0a1c-4d16-8efa-90b6b2c6b7bd admin admin] PATH_INFO: `/v3/projects/e0f30331baa040648754f13987b5d92b/users/73e826627cd14173ab653cb943421a5a/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.523752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:16.527315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.567180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e25ac6-0a1c-4d16-8efa-90b6b2c6b7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f30331baa040648754f13987b5d92b, user_id=73e826627cd14173ab653cb943421a5a, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:16.576214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e25ac6-0a1c-4d16-8efa-90b6b2c6b7bd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:16.626109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 855/1685] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:16 2022] PUT /identity/v3/projects/e0f30331baa040648754f13987b5d92b/users/73e826627cd14173ab653cb943421a5a/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:16.627561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1988e5-6721-4592-8109-03e1e2a22d1b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.627860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1988e5-6721-4592-8109-03e1e2a22d1b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.658298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1988e5-6721-4592-8109-03e1e2a22d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.711987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a64f6-58f3-4aa7-be1b-c45b78c05b6e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:16.765919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bf1988e5-6721-4592-8109-03e1e2a22d1b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:16.774142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf1988e5-6721-4592-8109-03e1e2a22d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:16.894970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:16.897570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.898569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:16.899998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:16.993070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 856/1686] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:16.994821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:16.995028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.007005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.043652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] MFA Rules not processed for user `73e826627cd14173ab653cb943421a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:17.045549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.076694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:17.133973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.135535 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.173255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 831/1687] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:17.173937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.203357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce48693-1609-48bb-849e-4068f86b8454 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.229331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.229331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.229331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.229331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.229785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:17.237563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.238966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:43:17.238966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Jun 15 14:43:17.238966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: Jun 15 14:43:17.239750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.255402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 857/1688] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:17.256426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.269799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcd160-ba47-4759-b8ff-8fe2629adacd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.291192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 832/1689] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:17.292159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.292300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.293102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.293393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.293777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.295260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.297306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.301138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:17.323663 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcafc7-9848-4f8d-a7ab-def69ea935b0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.345858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347939c-c220-41a7-ac54-67149f108209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.346570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347939c-c220-41a7-ac54-67149f108209 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.346844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347939c-c220-41a7-ac54-67149f108209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.347134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347939c-c220-41a7-ac54-67149f108209 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.355333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347939c-c220-41a7-ac54-67149f108209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.366190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347939c-c220-41a7-ac54-67149f108209 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.375260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 858/1690] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/projects => generated 409 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:17.376567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.458083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 833/1691] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:17.458083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432f3b8-83d1-4f78-937e-d7791b011de7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.458083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432f3b8-83d1-4f78-937e-d7791b011de7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.458083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432f3b8-83d1-4f78-937e-d7791b011de7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.460087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 834/1692] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:17.460915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432f3b8-83d1-4f78-937e-d7791b011de7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.553266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a7134-c851-4ff1-8a8e-c67797b23ba8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.638740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432f3b8-83d1-4f78-937e-d7791b011de7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.649582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e636-04b3-45c0-9501-133f8d1809fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.650641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e636-04b3-45c0-9501-133f8d1809fe admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.651441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e636-04b3-45c0-9501-133f8d1809fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.651829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e636-04b3-45c0-9501-133f8d1809fe admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.652851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e636-04b3-45c0-9501-133f8d1809fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.657435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e636-04b3-45c0-9501-133f8d1809fe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.657577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.658538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.658839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.659995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.660674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.663146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.688157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 859/1693] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/users => generated 311 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:17.689693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.690111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.690325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.746171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:17.750702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:17.758723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 835/1694] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/users => generated 286 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:17.759969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.821611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097dd1c-573b-4589-90fc-71accc3f3a01 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.842600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.843517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.843909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.844281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.845100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.852179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.864438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:17.895706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 836/1695] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:17.896641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.928192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3740a-9baf-4835-8f95-299c29d8b640 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:17.948406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:17.949394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:17.950324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:17.950927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:17.955591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 860/1696] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:17.956752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:17.956914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:17.964213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:17.997325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 837/1697] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:18.000107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.015082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36676dc-cac4-4fb6-bf71-5880543e9b63 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.031071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9becb-f852-4de0-865d-7f5a0134b1ba admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.036455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.042133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.042429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.042714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] PATH_INFO: `/v3/projects/ffa1f1862a924b9fb2b42a54b093494d/users/522f6eb69b8e4690a78532cca9041a2b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.051851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.062430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.063445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.063932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.064896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.066899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.089211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 838/1698] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:18.095090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.102861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa1f1862a924b9fb2b42a54b093494d, user_id=522f6eb69b8e4690a78532cca9041a2b, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.104760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:18.108748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.131745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44c926-5afa-4d90-8f1f-56d1b13c4e6f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.134553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 861/1699] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:17 2022] PUT /identity/v3/projects/ffa1f1862a924b9fb2b42a54b093494d/users/522f6eb69b8e4690a78532cca9041a2b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:18.140071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.258631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630780f1-0237-4f90-b676-451b83b50959 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.261799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.267012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1776c6-f1fd-4ec4-b55d-380b239e68d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.267993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.268116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1776c6-f1fd-4ec4-b55d-380b239e68d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.268295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1776c6-f1fd-4ec4-b55d-380b239e68d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.269810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.270210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.270322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1776c6-f1fd-4ec4-b55d-380b239e68d3 admin admin] PATH_INFO: `/v3/projects/e730a4e6489d4e7480445b8210e15566/users/36bff42e33854d2ca51b28a1c5118eba/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.287227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.305032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.331820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 839/1700] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:18.332810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.346439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1776c6-f1fd-4ec4-b55d-380b239e68d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e730a4e6489d4e7480445b8210e15566, user_id=36bff42e33854d2ca51b28a1c5118eba, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.354710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1776c6-f1fd-4ec4-b55d-380b239e68d3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.368354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09972bf3-a1e8-49f2-92f7-c86bf5e356b3 service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.381137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.382319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.383142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.383498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.384473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.386386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.387091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 862/1701] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:18 2022] PUT /identity/v3/projects/e730a4e6489d4e7480445b8210e15566/users/36bff42e33854d2ca51b28a1c5118eba/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:18.400657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.401338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.404475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.410299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 840/1702] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:18.411730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.460333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f0aa4-6b5c-42ea-82bb-d7f9f73979f0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:18.485587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:18.547203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] MFA Rules not processed for user `36bff42e33854d2ca51b28a1c5118eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:18.548390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:18.631521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd33a51-d02c-47ea-bec2-4b9afd3b6eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:18.632490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33a51-d02c-47ea-bec2-4b9afd3b6eca admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.632781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33a51-d02c-47ea-bec2-4b9afd3b6eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.633122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33a51-d02c-47ea-bec2-4b9afd3b6eca admin admin] PATH_INFO: `/v3/projects/ffa1f1862a924b9fb2b42a54b093494d/users/522f6eb69b8e4690a78532cca9041a2b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.639258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd33a51-d02c-47ea-bec2-4b9afd3b6eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa1f1862a924b9fb2b42a54b093494d, user_id=522f6eb69b8e4690a78532cca9041a2b, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:18.645400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd33a51-d02c-47ea-bec2-4b9afd3b6eca admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:18.689570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 841/1703] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:18 2022] PUT /identity/v3/projects/ffa1f1862a924b9fb2b42a54b093494d/users/522f6eb69b8e4690a78532cca9041a2b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:18.718709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:18.719037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:18.719287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:18.732657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 863/1704] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:18.747055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:18.749998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:18.803818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:18.809180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] MFA Rules not processed for user `522f6eb69b8e4690a78532cca9041a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:18.813555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:18.934144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a59869-009d-4de7-96ec-4eae7598a92a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.090607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 842/1705] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:19.108188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.109684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:19.152087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3204e-cb9f-4ac2-8f40-82efffb8b99e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.155145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3204e-cb9f-4ac2-8f40-82efffb8b99e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.155587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3204e-cb9f-4ac2-8f40-82efffb8b99e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.155977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3204e-cb9f-4ac2-8f40-82efffb8b99e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.238826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b11c-da71-418b-b774-a7483a0594e2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.242777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.243464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.243757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.244043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.281223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.282723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.318212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 843/1706] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:19.396566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3204e-cb9f-4ac2-8f40-82efffb8b99e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:19.398584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3204e-cb9f-4ac2-8f40-82efffb8b99e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:19.440348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 864/1707] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:19.852550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:19.854997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:19.875531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605f21c-2353-411e-ae43-0013266b1d64 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:19.901403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:19.902256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.903345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.903345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:19.991990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd60bd2-01d1-43ef-b6bf-9a8279ec8db5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:19.992172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd60bd2-01d1-43ef-b6bf-9a8279ec8db5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:19.992533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd60bd2-01d1-43ef-b6bf-9a8279ec8db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.047281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8cd60bd2-01d1-43ef-b6bf-9a8279ec8db5 None None] MFA Rules not processed for user `d292189b741b40b8859bfe9883b4c5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:20.048573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8cd60bd2-01d1-43ef-b6bf-9a8279ec8db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.058967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:20.121412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.123502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.163210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 844/1708] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:20.205213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:20.223163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 865/1709] 10.222.0.70 () {58 vars in 1015 bytes} [Wed Jun 15 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:20.326953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:20.328995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:20.378555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8a73b-41e0-4c90-862f-6cc190cdfe88 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:20.392204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:20.393955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:20.395337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:20.395863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:20.426796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:20.428572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:20.451732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 845/1710] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:21.134442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72cf0b-369c-454b-92d5-67b605c5ccde None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.135521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72cf0b-369c-454b-92d5-67b605c5ccde None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.135959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72cf0b-369c-454b-92d5-67b605c5ccde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.180736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.188703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.243658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7e72cf0b-369c-454b-92d5-67b605c5ccde None None] MFA Rules not processed for user `28ff37865a9b49ec8601140fca00a627`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:21.246699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e72cf0b-369c-454b-92d5-67b605c5ccde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.422794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64eff-ba4c-401d-aba4-1b71aa7cc41a service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.441125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.442000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.442296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.442588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] PATH_INFO: `/v3/users/91bc239ccbb047578e896bd984ea420b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.449199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91bc239ccbb047578e896bd984ea420b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.451767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:21.523180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 866/1711] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:21.529587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.531394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.533140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.634778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91bc239ccbb047578e896bd984ea420b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:21.670769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91bc239ccbb047578e896bd984ea420b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:21.690335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91bc239ccbb047578e896bd984ea420b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:21.696765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] MFA Rules not processed for user `ed7372fed3e14f3e98ca25d22a7c7a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:21.701540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.703584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91bc239ccbb047578e896bd984ea420b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:21.704267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91bc239ccbb047578e896bd984ea420b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:21.714467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 846/1712] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:21 2022] DELETE /identity/v3/users/91bc239ccbb047578e896bd984ea420b => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:21.714467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:21.714467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:21.879245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:21.914127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f49b1f-3bfd-4f1f-993d-57be8f467fe9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:21.919095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:21.921145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:21.921145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:21.921365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] PATH_INFO: `/v3/projects/c6d8b04fcde14efe8dae7d53b149196c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:21.926676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d8b04fcde14efe8dae7d53b149196c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:21.928654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.021086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 867/1713] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:22.022049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:22.027163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.070838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d8b04fcde14efe8dae7d53b149196c', 'request_id': 'req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:22.098231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d8b04fcde14efe8dae7d53b149196c', 'request_id': 'req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:22.108001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d8b04fcde14efe8dae7d53b149196c', 'request_id': 'req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:22.139373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] The token cache is being invalidate because project c6d8b04fcde14efe8dae7d53b149196c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:22.139653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:22.140429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6e6aabf1-8499-431e-a3dd-7b6a6f67e722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:22.142927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 847/1714] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:21 2022] DELETE /identity/v3/projects/c6d8b04fcde14efe8dae7d53b149196c => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:22.150476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030cf2d-385e-40a6-a5fa-2114f5c5ccc9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.150777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030cf2d-385e-40a6-a5fa-2114f5c5ccc9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.151113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030cf2d-385e-40a6-a5fa-2114f5c5ccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.196106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b030cf2d-385e-40a6-a5fa-2114f5c5ccc9 None None] MFA Rules not processed for user `1d58ac7f49454fd6a5b5eef6a1e8b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:22.202593 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b030cf2d-385e-40a6-a5fa-2114f5c5ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.204166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d72b8-7898-44e3-b245-66dde85cb7b0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:22.220326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:22.293516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 848/1715] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:22.295842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60b345-c4e0-4cec-85ee-c82bc9847fd0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.296374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60b345-c4e0-4cec-85ee-c82bc9847fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.296960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60b345-c4e0-4cec-85ee-c82bc9847fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.331807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4e60b345-c4e0-4cec-85ee-c82bc9847fd0 None None] MFA Rules not processed for user `ed7372fed3e14f3e98ca25d22a7c7a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:22.334383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4e60b345-c4e0-4cec-85ee-c82bc9847fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.449491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:22.451420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.451722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.452012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.457579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.498888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 849/1716] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:22.500823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.504436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.505094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.570098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:22.572928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.658900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:22.702755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:22.704705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b45e92-25f1-46ee-ade9-564a4411ad5a service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:22.737872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 868/1717] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:22.882319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 850/1718] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:22.882686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:22.883842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:22.883842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:22.948907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:22.951055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:22.994674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 869/1719] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:22.995402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.014043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3cd7b-ddd2-44fb-a699-e0e1a9123788 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.030123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.030879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.031110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.031394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.032149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:23.032468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.034208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.043973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.045824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.066118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 851/1720] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:23.067663 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.077778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc5e48-675d-4dc4-853a-2041c3afc795 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.088473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe695d6-d79d-4c4a-a2ce-b0cd3d5005be admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.101359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.102145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.102443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.102724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.103543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.104886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.105634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.107254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.107254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.107996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] PATH_INFO: `/v3/projects/459c99f4d26041ccbc68b1a0c660570a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.110813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=459c99f4d26041ccbc68b1a0c660570a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.113711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.184779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 852/1721] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:23 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:23.191407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.238089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459c99f4d26041ccbc68b1a0c660570a', 'request_id': 'req-4b34e69c-c116-4714-a24a-824775dab11b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.263733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459c99f4d26041ccbc68b1a0c660570a', 'request_id': 'req-4b34e69c-c116-4714-a24a-824775dab11b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.273230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459c99f4d26041ccbc68b1a0c660570a', 'request_id': 'req-4b34e69c-c116-4714-a24a-824775dab11b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.291100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] The token cache is being invalidate because project 459c99f4d26041ccbc68b1a0c660570a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:23.291534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.293086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.299101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 870/1722] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:23 2022] DELETE /identity/v3/projects/459c99f4d26041ccbc68b1a0c660570a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:23.299101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.312574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.368891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:23.480217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b199b-8334-4248-ad5d-f72d357f9597 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.525258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34e69c-c116-4714-a24a-824775dab11b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.550277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.552334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.552728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.553115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.557959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.705321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4840f1-ea17-44d5-a507-37ad585d0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.705321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4840f1-ea17-44d5-a507-37ad585d0759 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.705321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4840f1-ea17-44d5-a507-37ad585d0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.705321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4840f1-ea17-44d5-a507-37ad585d0759 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.705933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4840f1-ea17-44d5-a507-37ad585d0759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.709692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4840f1-ea17-44d5-a507-37ad585d0759 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.719879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.721688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.735109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 871/1723] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:23.753920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.760087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.774334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 853/1724] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:23 2022] POST /identity/v3/users => generated 302 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:23.776761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a325e-bb4d-4b16-8149-6c616bd98cd8 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.777596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a325e-bb4d-4b16-8149-6c616bd98cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.777910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a325e-bb4d-4b16-8149-6c616bd98cd8 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.786166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 854/1725] 10.222.0.57 () {54 vars in 912 bytes} [Wed Jun 15 14:43:23 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:23.792752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a325e-bb4d-4b16-8149-6c616bd98cd8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:23.798655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4e0a325e-bb4d-4b16-8149-6c616bd98cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.799042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad27ab-6adc-4959-9395-53706ae2177b service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.804761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.805886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.806281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.806669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] PATH_INFO: `/v3/users/76bbde0223de4b1883b288fdf9df8225` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.808149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bbde0223de4b1883b288fdf9df8225)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.810814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.840324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a325e-bb4d-4b16-8149-6c616bd98cd8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:23.850433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3335d-3bd3-40c3-8f97-a1198a936fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:23.851308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3335d-3bd3-40c3-8f97-a1198a936fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.851600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3335d-3bd3-40c3-8f97-a1198a936fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.853165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3335d-3bd3-40c3-8f97-a1198a936fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.855071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3335d-3bd3-40c3-8f97-a1198a936fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:23.859842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3335d-3bd3-40c3-8f97-a1198a936fb6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:23.882917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 855/1726] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:23.894846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:23.895134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:23.895400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:23.907213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bbde0223de4b1883b288fdf9df8225'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.910146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bbde0223de4b1883b288fdf9df8225'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.928392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bbde0223de4b1883b288fdf9df8225'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.929530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bbde0223de4b1883b288fdf9df8225'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.929791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] MFA Rules not processed for user `d292189b741b40b8859bfe9883b4c5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:23.931222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:23.953235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bbde0223de4b1883b288fdf9df8225'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:23.957744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:23.999677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 872/1727] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:23 2022] DELETE /identity/v3/users/76bbde0223de4b1883b288fdf9df8225 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:24.000662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:24.006605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:24.071260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 856/1728] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:24.112606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:24.114209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:24.114338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:24.131041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12148-3e5e-41b4-968d-b8986b0a3ae0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:24.171009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31505394-486d-4e4d-8091-f16a7eec1790 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:24.186937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:24.188023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.189362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.192282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.194007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:24.197258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:24.227257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 873/1729] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:24.228632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:24.257509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dae02d-160f-4491-91ce-fa1c2bf4c509 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:24.264394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bc6df-d643-4856-939b-d6082b9868ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:24.265653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc6df-d643-4856-939b-d6082b9868ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.266334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc6df-d643-4856-939b-d6082b9868ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.266722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc6df-d643-4856-939b-d6082b9868ad admin admin] PATH_INFO: `/v3/projects/3db9757a88eb43948fd6aad4884e0fa4/users/5acedb63ed9d448aa7ca327159ed26cd/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.286112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:24.287376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.287931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.288124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.295309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bc6df-d643-4856-939b-d6082b9868ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db9757a88eb43948fd6aad4884e0fa4, user_id=5acedb63ed9d448aa7ca327159ed26cd, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:24.306037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:24.306419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bc6df-d643-4856-939b-d6082b9868ad admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:24.307775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:24.346224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 874/1730] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:24 2022] PUT /identity/v3/projects/3db9757a88eb43948fd6aad4884e0fa4/users/5acedb63ed9d448aa7ca327159ed26cd/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:24.349294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 857/1731] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:24.349566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.350296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.350428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.383924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] MFA Rules not processed for user `5acedb63ed9d448aa7ca327159ed26cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:24.385450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:24.503120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:24.506157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 875/1732] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:24.527089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96f17-7fb2-468c-9d71-b5714ea88117 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:24.761608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:24.762444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:24.762895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:24.763307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:24.767327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:24.769420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:24.894954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 858/1733] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:24 2022] POST /identity/v3/projects => generated 403 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:24.899994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:24.925887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828dfde-d6f3-4dca-9b64-7821c0853013 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.097937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.100241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.100851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.102550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.103396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.105892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.117400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.119264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:25.147085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c636c-a150-4788-bff9-c8f92031a86b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.162108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 876/1734] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:24 2022] POST /identity/v3/users => generated 308 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:25.163087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.180276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.181307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b210ce-9476-4136-b6a1-5279e158a09c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.183658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.187051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.187637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] PATH_INFO: `/v3/projects/f0ea6519831d4e559f12808f0050709e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.188905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.189730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.190037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ea6519831d4e559f12808f0050709e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.190157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.190753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.191528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.193199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.197083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.219893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 877/1735] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:25.224144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.252177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748ec5b-9faa-49b4-8543-eb8ddba24857 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.267312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.268245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.268544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.268833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] PATH_INFO: `/v3/projects/602db8e9e9864b7781e1990f4b2efadb/users/c068b57a994b4388bc87703b52ef36c2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.324527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] RBAC: Authorizing `identity:create_grant(project_id=602db8e9e9864b7781e1990f4b2efadb, user_id=c068b57a994b4388bc87703b52ef36c2, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.333082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.345229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:25.357231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:25.363137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ea6519831d4e559f12808f0050709e', 'request_id': 'req-c13f66a3-0221-41c6-8e34-54187d648037'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:25.390101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 878/1736] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:25 2022] PUT /identity/v3/projects/602db8e9e9864b7781e1990f4b2efadb/users/c068b57a994b4388bc87703b52ef36c2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:25.392324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.417167 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ea6519831d4e559f12808f0050709e', 'request_id': 'req-c13f66a3-0221-41c6-8e34-54187d648037'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:25.429398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ea6519831d4e559f12808f0050709e', 'request_id': 'req-c13f66a3-0221-41c6-8e34-54187d648037'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:25.461367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] The token cache is being invalidate because project f0ea6519831d4e559f12808f0050709e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:25.462250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:25.463632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c13f66a3-0221-41c6-8e34-54187d648037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:25.469307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 859/1737] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:25 2022] DELETE /identity/v3/projects/f0ea6519831d4e559f12808f0050709e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:25.474421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aca73-61db-4a04-b0bc-b84519667e4f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.476069 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aca73-61db-4a04-b0bc-b84519667e4f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.477420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aca73-61db-4a04-b0bc-b84519667e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.532956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-121aca73-61db-4a04-b0bc-b84519667e4f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:25.556227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-121aca73-61db-4a04-b0bc-b84519667e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:25.650533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd45994-2356-435d-9645-55c41d64781d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.664101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.665975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.666266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.666547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.668518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.675641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.754237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 879/1738] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:25.759694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:25.764266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:25.839768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33207fc0-8fd7-4c4e-bbe1-066c78e9d36a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:25.858431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 860/1739] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:25.866889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac1a5-052e-468a-a61b-584c3d45e9ca None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.869855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac1a5-052e-468a-a61b-584c3d45e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.872984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac1a5-052e-468a-a61b-584c3d45e9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.877927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398c679-c170-4362-9a73-a7f271769e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:25.879259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398c679-c170-4362-9a73-a7f271769e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.879553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398c679-c170-4362-9a73-a7f271769e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.879895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398c679-c170-4362-9a73-a7f271769e6f admin admin] PATH_INFO: `/v3/projects/602db8e9e9864b7781e1990f4b2efadb/users/c068b57a994b4388bc87703b52ef36c2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.886909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398c679-c170-4362-9a73-a7f271769e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=602db8e9e9864b7781e1990f4b2efadb, user_id=c068b57a994b4388bc87703b52ef36c2, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:25.892496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398c679-c170-4362-9a73-a7f271769e6f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:25.912258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 880/1740] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:25 2022] PUT /identity/v3/projects/602db8e9e9864b7781e1990f4b2efadb/users/c068b57a994b4388bc87703b52ef36c2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:25.923655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76736a-b3a1-455d-8b99-9affbe275db6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:25.924566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76736a-b3a1-455d-8b99-9affbe275db6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:25.924747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76736a-b3a1-455d-8b99-9affbe275db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:25.939937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-686ac1a5-052e-468a-a61b-584c3d45e9ca None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:25.945022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-686ac1a5-052e-468a-a61b-584c3d45e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:26.026859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ca76736a-b3a1-455d-8b99-9affbe275db6 None None] MFA Rules not processed for user `c068b57a994b4388bc87703b52ef36c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:26.028198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ca76736a-b3a1-455d-8b99-9affbe275db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:26.175083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 861/1741] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:26.177014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462d295-f615-4525-8b37-06f352249ce6 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.178421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462d295-f615-4525-8b37-06f352249ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.180563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462d295-f615-4525-8b37-06f352249ce6 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.183402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 862/1742] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:43:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:43:26.185525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462d295-f615-4525-8b37-06f352249ce6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.211428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 881/1743] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:26.225035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.225143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.225954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.274339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462d295-f615-4525-8b37-06f352249ce6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.293873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.298764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.299120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.299408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.300215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:26.300534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.311809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.318864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:26.320471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:26.339338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 863/1744] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:26.340338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.342899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:26.444861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a603939-66ec-40ef-baf8-aa14da75833d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.470861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.493542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.494506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.496835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.582775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.592192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:26.599279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.657827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 864/1745] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:26.658786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.676910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 882/1746] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 15 14:43:26.687138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:26.707278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6b209-00a4-483f-8906-427d7208dacb service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:26.726811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:26.727688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:26.728127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:26.728598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:26.730415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:26.734478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:26.910084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 865/1747] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:26 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:26.914611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:26.937407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677779de-d2a7-4104-9259-fe864aa023ff None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.069158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.071461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.250263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.252307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.253064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.253379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.257172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.262725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.285402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6c237-ea01-4502-956d-a9a0f1b53f56 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.311485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.312716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.314683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.315198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] PATH_INFO: `/v3/users/36bff42e33854d2ca51b28a1c5118eba` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.319933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bff42e33854d2ca51b28a1c5118eba)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.334429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 883/1748] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:26 2022] POST /identity/v3/users => generated 289 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:27.334429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.334819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.343411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:27.648493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bff42e33854d2ca51b28a1c5118eba'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.649711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5ab95-4cc4-4873-9492-d04540017026 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.673247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:27.673992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:27.674311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:27.674568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:27.718188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:27.719986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bff42e33854d2ca51b28a1c5118eba'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.725808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:27.755445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bff42e33854d2ca51b28a1c5118eba'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.785435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 884/1749] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:27.786417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.809071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bff42e33854d2ca51b28a1c5118eba'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.810760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bff42e33854d2ca51b28a1c5118eba'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:27.834209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2dca4-844f-4c2c-8a25-d9b6cd393f27 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:27.838933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 866/1750] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:27 2022] DELETE /identity/v3/users/36bff42e33854d2ca51b28a1c5118eba => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:27.840103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:27.845991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.106453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.114652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.114986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.115297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] PATH_INFO: `/v3/users/5acedb63ed9d448aa7ca327159ed26cd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.123646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5acedb63ed9d448aa7ca327159ed26cd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.125634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.192486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:28.198270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a4865-5594-4cbf-812b-cb691beecc02 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.215461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b68de-3f26-4efb-acf9-055ab04bbedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.216303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b68de-3f26-4efb-acf9-055ab04bbedd admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.216634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b68de-3f26-4efb-acf9-055ab04bbedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.216929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b68de-3f26-4efb-acf9-055ab04bbedd admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.217720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b68de-3f26-4efb-acf9-055ab04bbedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.222751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b68de-3f26-4efb-acf9-055ab04bbedd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.250951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:28.269991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 867/1751] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:28.272385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.272684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.273011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.328536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:28.336713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.349622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5acedb63ed9d448aa7ca327159ed26cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.353038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5acedb63ed9d448aa7ca327159ed26cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.387365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 868/1752] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:28.398718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5acedb63ed9d448aa7ca327159ed26cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.399282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.401162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5acedb63ed9d448aa7ca327159ed26cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.409284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.438411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-152b6dd0-df70-460b-89be-6868ea8a5900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5acedb63ed9d448aa7ca327159ed26cd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:28.486672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3363f-0a1d-4dc8-b193-f15e044bd2e9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.505579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.506617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.507159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.507267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.511564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.521686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.535090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 885/1753] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:27 2022] DELETE /identity/v3/users/5acedb63ed9d448aa7ca327159ed26cd => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:28.542105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.542424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.546172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.586153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 869/1754] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:28.588939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.623473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:28.630461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.636982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb7a13-1d33-468d-89c0-b3659658598f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:28.827331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441b8e-3502-4810-9c82-7d52e1eb804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:28.827331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441b8e-3502-4810-9c82-7d52e1eb804d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.827331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441b8e-3502-4810-9c82-7d52e1eb804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.827331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441b8e-3502-4810-9c82-7d52e1eb804d admin admin] PATH_INFO: `/v3/projects/32bed34e68944ec3b11676de475afe75/users/78e0bd33c7414863bc6f719c581e5829/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:28.898764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da441b8e-3502-4810-9c82-7d52e1eb804d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bed34e68944ec3b11676de475afe75, user_id=78e0bd33c7414863bc6f719c581e5829, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:28.901691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da441b8e-3502-4810-9c82-7d52e1eb804d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:28.935734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 886/1755] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:28.938781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:28.945945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:28.961933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 870/1756] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:28 2022] PUT /identity/v3/projects/32bed34e68944ec3b11676de475afe75/users/78e0bd33c7414863bc6f719c581e5829/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:28.969485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26485365-09a2-435a-8247-8f8ff2318b5d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:28.969797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26485365-09a2-435a-8247-8f8ff2318b5d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:28.970091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26485365-09a2-435a-8247-8f8ff2318b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.044527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-26485365-09a2-435a-8247-8f8ff2318b5d None None] MFA Rules not processed for user `ed7372fed3e14f3e98ca25d22a7c7a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:29.046109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-26485365-09a2-435a-8247-8f8ff2318b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.165564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01815e-2425-4f49-9f4b-c17cd2a022ea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.228708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 871/1757] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:29.230264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.231048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.231414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.250476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:29.260712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.262042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.262524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.264320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.446967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.448507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.474979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 887/1758] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:29.475198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.476459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.509264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04723ea4-6391-49ae-936b-3410d5751135 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.519843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3b48c-f928-47bd-8dcc-49cc7da6fd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.520526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b48c-f928-47bd-8dcc-49cc7da6fd26 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.520813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b48c-f928-47bd-8dcc-49cc7da6fd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.521090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b48c-f928-47bd-8dcc-49cc7da6fd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.542027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3b48c-f928-47bd-8dcc-49cc7da6fd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.545874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3b48c-f928-47bd-8dcc-49cc7da6fd26 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.571977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 888/1759] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:29.573690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.582403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.582931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.623453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] MFA Rules not processed for user `28ff37865a9b49ec8601140fca00a627`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:29.624985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.678884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] MFA Rules not processed for user `78e0bd33c7414863bc6f719c581e5829`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:29.680122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:29.745784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 889/1760] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:29.830519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:29.855015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 872/1761] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 625 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:29.860529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.881582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e691e83-2fe9-4d6c-908e-0aa5bf31967c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:29.890333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:29.891677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:29.892104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:29.892519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:29.893487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:29.896732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:29.954818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 890/1762] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:29 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:29.956475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:29.972920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fc8ed-e5cb-4f09-8d6a-1091db203b15 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.092260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.094574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.095009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.095769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.103197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.105493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.162009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 873/1763] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:29 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:30.166859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.194236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0748-be47-4a7c-8831-847224ed7120 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.205869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.206848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.207199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.207602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.208367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.213333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.224100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.226280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.248272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 891/1764] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:30.249488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06473c-c4a8-4be7-a934-8922df57ac52 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.252310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.265889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.268286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.268586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.269598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] PATH_INFO: `/v3/projects/3db9757a88eb43948fd6aad4884e0fa4` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.274334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db9757a88eb43948fd6aad4884e0fa4)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.277479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8298e5-c700-4bd5-a1d6-1471f34f3478 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.279003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.290863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.293112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.293740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.294865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] PATH_INFO: `/v3/projects/98b17fddfe044d889a3d5e771f25da51/users/eba74be9499c4595ae5b2b9bd451b4bd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.346988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b17fddfe044d889a3d5e771f25da51, user_id=eba74be9499c4595ae5b2b9bd451b4bd, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.351396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.420260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 892/1765] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:30 2022] PUT /identity/v3/projects/98b17fddfe044d889a3d5e771f25da51/users/eba74be9499c4595ae5b2b9bd451b4bd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:30.424959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.477653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db9757a88eb43948fd6aad4884e0fa4', 'request_id': 'req-ee90d1ea-3a23-458a-9754-9ef3e68944a1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.533496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db9757a88eb43948fd6aad4884e0fa4', 'request_id': 'req-ee90d1ea-3a23-458a-9754-9ef3e68944a1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.555034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db9757a88eb43948fd6aad4884e0fa4', 'request_id': 'req-ee90d1ea-3a23-458a-9754-9ef3e68944a1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.590291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:30.590695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] The token cache is being invalidate because project 3db9757a88eb43948fd6aad4884e0fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:30.592879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.594036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.597543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 874/1766] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/projects/3db9757a88eb43948fd6aad4884e0fa4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:30.601311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.605858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.643829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec3810-7402-46bc-849a-e67226bfa43f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.796874 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90d1ea-3a23-458a-9754-9ef3e68944a1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.807538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.808443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.808726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.809011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] PATH_INFO: `/v3/users/c068b57a994b4388bc87703b52ef36c2` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.814950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c068b57a994b4388bc87703b52ef36c2)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.817493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.827403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.828107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.828403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.828693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.829447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.838777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:30.848540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:30.871945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 893/1767] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:30.884527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:30.887759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:30.943484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee90e4-6f54-41f4-8282-4efabbeb5b81 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:30.959485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:30.961298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:30.961597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:30.962177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] PATH_INFO: `/v3/projects/e730a4e6489d4e7480445b8210e15566` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:30.964265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c068b57a994b4388bc87703b52ef36c2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:30.968024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e730a4e6489d4e7480445b8210e15566)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:30.968384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.038123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c068b57a994b4388bc87703b52ef36c2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.088127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c068b57a994b4388bc87703b52ef36c2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.120950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c068b57a994b4388bc87703b52ef36c2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.122058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c068b57a994b4388bc87703b52ef36c2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.164027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 875/1768] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/users/c068b57a994b4388bc87703b52ef36c2 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:31.164766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.183782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.273642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e730a4e6489d4e7480445b8210e15566', 'request_id': 'req-478831f4-377c-41f0-bf97-03fd9f54dbc0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.300153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e730a4e6489d4e7480445b8210e15566', 'request_id': 'req-478831f4-377c-41f0-bf97-03fd9f54dbc0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.343007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e730a4e6489d4e7480445b8210e15566', 'request_id': 'req-478831f4-377c-41f0-bf97-03fd9f54dbc0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.388973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] The token cache is being invalidate because project e730a4e6489d4e7480445b8210e15566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:31.389279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.391611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-478831f4-377c-41f0-bf97-03fd9f54dbc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:31.402547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 894/1769] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:30 2022] DELETE /identity/v3/projects/e730a4e6489d4e7480445b8210e15566 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:31.408924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.409418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.412151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.493017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:31.500550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.584928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471775b-0a4b-4bb7-af43-4098eec429da admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.715754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:31.761637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 895/1770] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:31.762652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:31.765398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a724faa-ccbd-4842-863b-12712cc31f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.766616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a724faa-ccbd-4842-863b-12712cc31f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.767374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a724faa-ccbd-4842-863b-12712cc31f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.767374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a724faa-ccbd-4842-863b-12712cc31f72 admin admin] PATH_INFO: `/v3/projects/98b17fddfe044d889a3d5e771f25da51/users/eba74be9499c4595ae5b2b9bd451b4bd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.769466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a724faa-ccbd-4842-863b-12712cc31f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b17fddfe044d889a3d5e771f25da51, user_id=eba74be9499c4595ae5b2b9bd451b4bd, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.772263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a724faa-ccbd-4842-863b-12712cc31f72 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:31.772408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.795310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001407d-60dd-487a-8fce-f7079c089086 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:31.817042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 876/1771] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:31 2022] PUT /identity/v3/projects/98b17fddfe044d889a3d5e771f25da51/users/eba74be9499c4595ae5b2b9bd451b4bd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:31.820564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67101-272d-4146-a31f-2d0b6fcb58c4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.822114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67101-272d-4146-a31f-2d0b6fcb58c4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.822492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67101-272d-4146-a31f-2d0b6fcb58c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.858933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fbe67101-272d-4146-a31f-2d0b6fcb58c4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:31.860345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fbe67101-272d-4146-a31f-2d0b6fcb58c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:31.898036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:31.956524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:31.971614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:31.971967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:31.972386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] PATH_INFO: `/v3/users/73e826627cd14173ab653cb943421a5a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:31.982102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e826627cd14173ab653cb943421a5a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:31.988193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.011022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 877/1772] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:32.015730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af617571-94cd-487d-b564-bb9537ca2eb3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.016277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af617571-94cd-487d-b564-bb9537ca2eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.018241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af617571-94cd-487d-b564-bb9537ca2eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.122028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e826627cd14173ab653cb943421a5a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:32.123855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e826627cd14173ab653cb943421a5a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:32.132926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-af617571-94cd-487d-b564-bb9537ca2eb3 None None] MFA Rules not processed for user `eba74be9499c4595ae5b2b9bd451b4bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:32.134381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-af617571-94cd-487d-b564-bb9537ca2eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.136575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e826627cd14173ab653cb943421a5a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:32.137559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e826627cd14173ab653cb943421a5a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:32.150230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e826627cd14173ab653cb943421a5a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:32.199237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 896/1773] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:31 2022] DELETE /identity/v3/users/73e826627cd14173ab653cb943421a5a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:32.199237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.199237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.385895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 878/1774] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:32.388249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.388512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.388720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.389726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf922f-afc7-47a6-bdca-d974fbd63e98 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.400427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.401532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.402470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.402956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.404559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:32.405093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.406965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.410792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:32.412592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.425118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 879/1775] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:32.426284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.427883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.459807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 897/1776] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:32.472391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.477887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.506045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:32.567343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fac258-3060-4e90-951b-a75741cdd465 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.573682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.574461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.574818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.575257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.581078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:32.645178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e30d1-bff5-46be-bf53-8ecb1ef87410 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.653070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.653899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.654277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.654709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.672914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.674706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.696785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 898/1777] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:32.697935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.728432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45cd65-4087-4e4c-8cdf-e0c13633e3c2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:32.756295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:32.757200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:32.757718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:32.760087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:32.761118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.777207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:32.780424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.785742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:32.812542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 880/1778] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:32.839226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 899/1779] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:32.839639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:32.850227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:32.862973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6a9f8-1427-487a-84d5-07a4aab289f1 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.039384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.040150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.040597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.041585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.042299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.047571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.111530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.112598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 881/1780] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:32 2022] POST /identity/v3/users => generated 291 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:33.127365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.131672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.133881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef54ee-19f6-4deb-b817-39abb3e5a303 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.152142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.153017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.153380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.153741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.154490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.156099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.163502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ed930-655e-4ac4-ba73-be89501a02c9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.172877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.174473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.174604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.175104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] PATH_INFO: `/v3/projects/602db8e9e9864b7781e1990f4b2efadb` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.176137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 900/1781] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:33.177371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=602db8e9e9864b7781e1990f4b2efadb)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.180016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.180554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.193679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc0e1c-d3fe-4d1c-bb4f-64adf6105220 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.223539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.224091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.224322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.226878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.227747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.230628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.257025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 901/1782] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:33.267120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.289439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128bc4b-58fd-41e8-8ae0-f2d8f17283cf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.304737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '602db8e9e9864b7781e1990f4b2efadb', 'request_id': 'req-394bb128-d0bd-4350-955f-40f244184fce'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.312637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '602db8e9e9864b7781e1990f4b2efadb', 'request_id': 'req-394bb128-d0bd-4350-955f-40f244184fce'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.320579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '602db8e9e9864b7781e1990f4b2efadb', 'request_id': 'req-394bb128-d0bd-4350-955f-40f244184fce'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.390883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] The token cache is being invalidate because project 602db8e9e9864b7781e1990f4b2efadb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:33.391195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.396169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-394bb128-d0bd-4350-955f-40f244184fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:33.412837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 882/1783] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:33 2022] DELETE /identity/v3/projects/602db8e9e9864b7781e1990f4b2efadb => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:33.412837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e86eef-5117-4ed5-bac4-4ad6d4d2a3ce None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.412837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e86eef-5117-4ed5-bac4-4ad6d4d2a3ce None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.412837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e86eef-5117-4ed5-bac4-4ad6d4d2a3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.463329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29e0ae-3111-4fa8-a339-3892579e3f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.464268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29e0ae-3111-4fa8-a339-3892579e3f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.464558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29e0ae-3111-4fa8-a339-3892579e3f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.464842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29e0ae-3111-4fa8-a339-3892579e3f66 admin admin] PATH_INFO: `/v3/projects/3e81cc8aeeff4d299e5c3c51ab0c138b/users/8947d5e075e345029706aa45253e055a/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.470585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e0e86eef-5117-4ed5-bac4-4ad6d4d2a3ce None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:33.472624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e0e86eef-5117-4ed5-bac4-4ad6d4d2a3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.500879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 883/1784] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:33.503986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.504288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.504571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.509951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29e0ae-3111-4fa8-a339-3892579e3f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e81cc8aeeff4d299e5c3c51ab0c138b, user_id=8947d5e075e345029706aa45253e055a, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.514666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29e0ae-3111-4fa8-a339-3892579e3f66 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.540788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:33.542257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 902/1785] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:33 2022] PUT /identity/v3/projects/3e81cc8aeeff4d299e5c3c51ab0c138b/users/8947d5e075e345029706aa45253e055a/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:33.542673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.556759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.557074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.557403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.631977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] MFA Rules not processed for user `8947d5e075e345029706aa45253e055a`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:33.636336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.657077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 884/1786] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:33.661497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.673864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababbfaf-a9fc-4048-91c9-ec3ed3990c91 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.682955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.683636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.684053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.685262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.685884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:33.686555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.689023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.697899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:33.700266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:33.709840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 885/1787] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:43:33.712941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.726814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3259-b575-4f72-9814-292150274caf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.734859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.735533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.735896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.736904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.737290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.740074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.743270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 903/1788] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:33.749081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.750359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:33.779818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd10763-d969-48dd-b65e-ad5feb643fd3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.801856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 886/1789] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:33.801960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.806017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.806767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.807416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.807719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.808443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.813859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:33.868675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 904/1790] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:33.873255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:33.917985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b472e-88d8-4090-906e-2d0c4c3b5693 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.983204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd60cdb-91f8-485c-b077-e8bfe82480d3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:33.989639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:33.990369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:33.990704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:33.991023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:33.991958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:33.993697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.000462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.001538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.001649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.001955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.002680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.004655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.025738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 905/1791] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:34.029976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.034595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 887/1792] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:33 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:34.038501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.043023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce9ce6-d819-4228-a152-5c4f5bb098c0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.050585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.050745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.051624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.052181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.052813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.054658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.065905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78174e-c333-4c6a-9527-e935c0144efa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.066328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 906/1793] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:34.070737 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.083485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8c64-a856-4e18-a82c-d5a20e0b784e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.090012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.091033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.091870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.092195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] PATH_INFO: `/v3/projects/1b78473b6c30427fb258c866c9cfb0d1/users/c59a2d16c76144899139efc64dc3d1bc/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.099141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.101046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.101046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.101046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.102502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.104182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.110831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b78473b6c30427fb258c866c9cfb0d1, user_id=c59a2d16c76144899139efc64dc3d1bc, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.113963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.118625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 888/1794] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:34.123612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.143321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 907/1795] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:34 2022] PUT /identity/v3/projects/1b78473b6c30427fb258c866c9cfb0d1/users/c59a2d16c76144899139efc64dc3d1bc/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:34.147349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.147474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea0b39-1c0d-463e-95e2-26c873c884d4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.281248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.282269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.283075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.283075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.283982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.285800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.288253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0d9e8-fc66-4c56-9be8-fe1c5d88945f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.288902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:34.301073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 889/1796] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:34.305043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.316347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.317308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.317609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.317969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.318620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc48b4-3147-4105-96e4-cce2e579f0f6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.318742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.320579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.324090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798189b-63ff-4fe3-b6c1-9bad70171c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.326380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798189b-63ff-4fe3-b6c1-9bad70171c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.326754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798189b-63ff-4fe3-b6c1-9bad70171c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.327027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798189b-63ff-4fe3-b6c1-9bad70171c09 admin admin] PATH_INFO: `/v3/projects/da9c99432ab44a84ba98417ae23ce392/users/441bd4602a42415eb332b791d4aa7d69/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.365694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 908/1797] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:34.368891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.373734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798189b-63ff-4fe3-b6c1-9bad70171c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9c99432ab44a84ba98417ae23ce392, user_id=441bd4602a42415eb332b791d4aa7d69, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.380547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798189b-63ff-4fe3-b6c1-9bad70171c09 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.396014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33196239-8521-4c5e-a3d7-9fcfb8c502b4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.409158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad56a31-4402-42c2-b79d-e8d154db5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.411997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56a31-4402-42c2-b79d-e8d154db5135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.412293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56a31-4402-42c2-b79d-e8d154db5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.412582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56a31-4402-42c2-b79d-e8d154db5135 admin admin] PATH_INFO: `/v3/projects/1b78473b6c30427fb258c866c9cfb0d1/users/c59a2d16c76144899139efc64dc3d1bc/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.414282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 890/1798] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:34 2022] PUT /identity/v3/projects/da9c99432ab44a84ba98417ae23ce392/users/441bd4602a42415eb332b791d4aa7d69/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:34.429540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.429961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.431415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad56a31-4402-42c2-b79d-e8d154db5135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b78473b6c30427fb258c866c9cfb0d1, user_id=c59a2d16c76144899139efc64dc3d1bc, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.431724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.440881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad56a31-4402-42c2-b79d-e8d154db5135 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.464689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:34.500593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 909/1799] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:34 2022] PUT /identity/v3/projects/1b78473b6c30427fb258c866c9cfb0d1/users/c59a2d16c76144899139efc64dc3d1bc/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:34.504608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.505341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.505632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.517210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] MFA Rules not processed for user `441bd4602a42415eb332b791d4aa7d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:34.520076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:34.571369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] MFA Rules not processed for user `c59a2d16c76144899139efc64dc3d1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:34.577096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:34.710570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 891/1800] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:34.719925 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.762550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 910/1801] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:34.777370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:34.779097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:34.906639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484cd8c-bab3-4796-995c-0d15ef68bc40 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:34.942436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:34.943150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:34.943424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:34.943711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:34.944546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:34.973858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:34.993449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a794c7-d298-44fc-b837-d167a6c74ac2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.010568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.011895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.012321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.012498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.035782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.043155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.053719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 892/1802] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:34 2022] POST /identity/v3/projects => generated 365 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:35.056927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.065054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 911/1803] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:35.168763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.171325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.190414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:35.218988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3844fdb-0791-4c73-9046-702f9a89af5c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.238841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.241891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.243018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.243799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] PATH_INFO: `/v3/projects/e0f30331baa040648754f13987b5d92b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.243923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cc8ae-a85f-4a44-b960-f456b9497d77 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.245954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f30331baa040648754f13987b5d92b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.248718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.257728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.261582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.262172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.262658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.263672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.269013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.303124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 893/1804] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:35 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:35.304449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.349348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f30331baa040648754f13987b5d92b', 'request_id': 'req-389ef632-7d03-48ba-b964-09a0f59b2d54'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.361882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f30331baa040648754f13987b5d92b', 'request_id': 'req-389ef632-7d03-48ba-b964-09a0f59b2d54'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.388626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f30331baa040648754f13987b5d92b', 'request_id': 'req-389ef632-7d03-48ba-b964-09a0f59b2d54'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.411848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] The token cache is being invalidate because project e0f30331baa040648754f13987b5d92b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:35.412224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.412944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.415801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 912/1805] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:35 2022] DELETE /identity/v3/projects/e0f30331baa040648754f13987b5d92b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:35.416638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.418097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.437937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:35.443521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2a191-f414-408e-a7e1-a6ff25d0dd03 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.523889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.524752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.525426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.525426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.528417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.561385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ef632-7d03-48ba-b964-09a0f59b2d54 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.567483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.568629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.569136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.569622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.570797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.572699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.591902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 913/1806] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:35.591902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.593741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:35.616403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b9aa-f731-4c9b-8920-52e2622c3297 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.624265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.625690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.626125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.626682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] PATH_INFO: `/v3/users/522f6eb69b8e4690a78532cca9041a2b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.629036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522f6eb69b8e4690a78532cca9041a2b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.632521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.664581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.667190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.698836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 894/1807] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:35.700207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.718476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d53430-4abc-4d8c-aa26-fb423db83af8 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:35.725618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:35.726771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:35.727263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:35.727790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] PATH_INFO: `/v3/projects/ce80155917b04aa3b76ea4c87ff364a3/users/754fe863dac243e7a99e11aa6de0b722/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:35.756008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce80155917b04aa3b76ea4c87ff364a3, user_id=754fe863dac243e7a99e11aa6de0b722, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:35.759578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:35.812003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 895/1808] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:35 2022] PUT /identity/v3/projects/ce80155917b04aa3b76ea4c87ff364a3/users/754fe863dac243e7a99e11aa6de0b722/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:35.817465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:35.901382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522f6eb69b8e4690a78532cca9041a2b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.902102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522f6eb69b8e4690a78532cca9041a2b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.913501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522f6eb69b8e4690a78532cca9041a2b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.914449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522f6eb69b8e4690a78532cca9041a2b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.929680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522f6eb69b8e4690a78532cca9041a2b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:35.934738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:36.000319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 914/1809] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:35 2022] DELETE /identity/v3/users/522f6eb69b8e4690a78532cca9041a2b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:36.019982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f82582-5c33-4134-8667-618513a264f5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.160654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ade9e-9db6-4ed3-966b-cd53d7e5397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.161367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ade9e-9db6-4ed3-966b-cd53d7e5397e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.163814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ade9e-9db6-4ed3-966b-cd53d7e5397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.164131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ade9e-9db6-4ed3-966b-cd53d7e5397e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.164904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ade9e-9db6-4ed3-966b-cd53d7e5397e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.169945 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ade9e-9db6-4ed3-966b-cd53d7e5397e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.193888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 896/1810] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:36.198167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:36.199941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.218891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b6fa3-8cb6-4080-80bd-f071a91da114 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.225512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.226884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.227314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.227696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] PATH_INFO: `/v3/projects/ce80155917b04aa3b76ea4c87ff364a3/users/754fe863dac243e7a99e11aa6de0b722/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.236819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce80155917b04aa3b76ea4c87ff364a3, user_id=754fe863dac243e7a99e11aa6de0b722, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.236819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.249750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 915/1811] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:36 2022] PUT /identity/v3/projects/ce80155917b04aa3b76ea4c87ff364a3/users/754fe863dac243e7a99e11aa6de0b722/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:36.259969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.260594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.261050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.306199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] MFA Rules not processed for user `754fe863dac243e7a99e11aa6de0b722`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:36.307735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.400210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 897/1812] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:36.412548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:36.414502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.511256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0150dd-cd11-4c65-ada0-92e11a3aa818 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.516262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.517638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.517973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.518268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.550333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.552060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.587431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 916/1813] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:36.726873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:36.738161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c574b9-7445-4fc9-b570-a31b5cf0dd82 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:36.742202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:36.742978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:36.743342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:36.743699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:36.746023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:36.815408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:36.829184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:36.831027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f407c-f14d-4890-b311-c15db837cd2e service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:36.843426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 898/1814] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:37.010312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.011902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:37.024238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:37.110340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6a9eb-3e8c-477b-a71d-5c8b6a6fea9d service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.115055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.115922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.116209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.116489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] PATH_INFO: `/v3/projects/ffa1f1862a924b9fb2b42a54b093494d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.117832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa1f1862a924b9fb2b42a54b093494d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.119719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.166727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa1f1862a924b9fb2b42a54b093494d', 'request_id': 'req-531d7fc1-a361-45cd-a429-0d35574128fe'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:37.175812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa1f1862a924b9fb2b42a54b093494d', 'request_id': 'req-531d7fc1-a361-45cd-a429-0d35574128fe'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:37.181593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa1f1862a924b9fb2b42a54b093494d', 'request_id': 'req-531d7fc1-a361-45cd-a429-0d35574128fe'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:37.194310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] The token cache is being invalidate because project ffa1f1862a924b9fb2b42a54b093494d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:37.194613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:37.195273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-531d7fc1-a361-45cd-a429-0d35574128fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:37.196878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 917/1815] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:37 2022] DELETE /identity/v3/projects/ffa1f1862a924b9fb2b42a54b093494d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:37.201848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d1c2-3297-46b6-be56-50238af476de None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.202333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d1c2-3297-46b6-be56-50238af476de None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.202558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d1c2-3297-46b6-be56-50238af476de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.220792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6869d1c2-3297-46b6-be56-50238af476de None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:37.222298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6869d1c2-3297-46b6-be56-50238af476de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:37.296665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 899/1816] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:37.305260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.305936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.306955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.333010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:37.335162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:37.345428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 918/1817] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:37.350582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869d1c2-3297-46b6-be56-50238af476de None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.368266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869d1c2-3297-46b6-be56-50238af476de None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.374837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.375570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.376433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.376433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.376904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:37.377253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.379125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.406176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 900/1818] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:37.410214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.422706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f2068-5e96-43dc-b2f9-ebfd688536dc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.428497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.429511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.430014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.430469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.431640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.434209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.476628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 919/1819] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:37 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:37.477967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.489335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade99ce-579b-4c28-89d5-8e09c798b5ec admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.548619 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.549337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.549677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.549970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.550691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.552620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.575882 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 901/1820] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:37 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:37.577165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.588656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea579-f5d0-4249-ac1b-5f8f9938bda8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.594356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.595049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.595336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.595615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.596365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.597997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.609318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 920/1821] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:37.613486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.624951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77094-8d2b-4bcd-9bed-ab4d7f305764 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.629909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22973a-ce46-4ef9-9306-83a91f159dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.630526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22973a-ce46-4ef9-9306-83a91f159dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.630821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22973a-ce46-4ef9-9306-83a91f159dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.631109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22973a-ce46-4ef9-9306-83a91f159dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.631853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22973a-ce46-4ef9-9306-83a91f159dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.633447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22973a-ce46-4ef9-9306-83a91f159dd1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.643886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 902/1822] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:37.647977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.658316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9bf32-c7b8-451f-887f-dbeb2aa9d178 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.662832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.663712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.664005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.664288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] PATH_INFO: `/v3/projects/b3f7d60debd24e58bda93b231bc2fef2/users/abcb74471e974a7baed9b1335f0f109b/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.679845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f7d60debd24e58bda93b231bc2fef2, user_id=abcb74471e974a7baed9b1335f0f109b, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.682529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.695292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 921/1823] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:37 2022] PUT /identity/v3/projects/b3f7d60debd24e58bda93b231bc2fef2/users/abcb74471e974a7baed9b1335f0f109b/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:37.706801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.706801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.707184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.736828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] MFA Rules not processed for user `abcb74471e974a7baed9b1335f0f109b`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:37.738096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:37.785783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:37.808169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 903/1824] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:37.819844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:37.821627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:37.857351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:37.885452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eec1b4-72ab-4a7a-8722-c678cfe9084f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:37.891461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d2fd4-b0f5-4b5e-91e9-814617f19e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:37.892166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d2fd4-b0f5-4b5e-91e9-814617f19e5a service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:37.892456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d2fd4-b0f5-4b5e-91e9-814617f19e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:37.892735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d2fd4-b0f5-4b5e-91e9-814617f19e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:37.902404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d2fd4-b0f5-4b5e-91e9-814617f19e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:37.905074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d2fd4-b0f5-4b5e-91e9-814617f19e5a service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:37.923328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 922/1825] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:37.948148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:38.114705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787fa7b-ec19-4321-8578-2caabaf6a711 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:38.137553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:38.140734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:38.141335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:38.141899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] PATH_INFO: `/v3/users/abcb74471e974a7baed9b1335f0f109b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:38.143679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abcb74471e974a7baed9b1335f0f109b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:38.146675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:38.220559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abcb74471e974a7baed9b1335f0f109b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:38.238140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abcb74471e974a7baed9b1335f0f109b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:38.248801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abcb74471e974a7baed9b1335f0f109b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:38.261029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abcb74471e974a7baed9b1335f0f109b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:38.261899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abcb74471e974a7baed9b1335f0f109b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:38.271451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 904/1826] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:37 2022] DELETE /identity/v3/users/abcb74471e974a7baed9b1335f0f109b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:38.276280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:38.276441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:38.276720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:38.306653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:38.310817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:38.413081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 923/1827] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:38.564505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:38.570770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:38.779590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04100981-fa95-4d13-be9d-55ee0cad8019 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:38.791701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:38.792389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:38.792680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:38.792961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:38.811463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:38.813651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:38.837733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 905/1828] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:39.195867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:39.198539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:39.353308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881c59-471e-44e8-b736-5c33e087f716 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:39.358093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:39.358851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:39.359208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:39.359563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:39.361396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:39.441008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:39.442884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a45aa-49b4-48ca-a135-8f9defc19167 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:39.462336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 924/1829] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:39.865183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:39.866756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:39.882501 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4cdf7-55f4-4483-b68e-bb8d9c0e045f service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:39.906968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:39.909504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:39.909963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:39.910968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] PATH_INFO: `/v3/projects/b3f7d60debd24e58bda93b231bc2fef2` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:39.913349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f7d60debd24e58bda93b231bc2fef2)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:39.916272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:39.924138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:40.008234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f7d60debd24e58bda93b231bc2fef2', 'request_id': 'req-6141b6e9-2934-482b-8940-7b887b135b24'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:40.016998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f7d60debd24e58bda93b231bc2fef2', 'request_id': 'req-6141b6e9-2934-482b-8940-7b887b135b24'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:40.024760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f7d60debd24e58bda93b231bc2fef2', 'request_id': 'req-6141b6e9-2934-482b-8940-7b887b135b24'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:40.043744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] The token cache is being invalidate because project b3f7d60debd24e58bda93b231bc2fef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:40.044563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:40.045467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6141b6e9-2934-482b-8940-7b887b135b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:40.047528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 906/1830] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:39 2022] DELETE /identity/v3/projects/b3f7d60debd24e58bda93b231bc2fef2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:40.062367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.062367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.062367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.082922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:40.084254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:40.120291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:40.183854 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 925/1831] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:40.194376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.194579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.194850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.229351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:40.231933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:40.248036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 907/1832] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:40.252592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.269084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9eed-ef00-4f95-ad22-baf9709c3221 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.274419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.275357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.275696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.275985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.276796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:40.277206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.280113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.301976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 926/1833] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:40.302927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.317960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dcb2cd-fa9d-4c62-bfe1-922c6a718f7b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.329316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.330518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.330930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.331346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.333693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.337928 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.380226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 908/1834] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:40.383414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.395528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee911-94ec-46aa-81e8-3346d4351d9b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.480284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.481359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.482722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.483218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.484733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.487176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.533943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 927/1835] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:40.534942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.554963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b49e-0144-4d34-bc6d-4189ce179317 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.570076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.570843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.572286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.572689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.574310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.580278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.597557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 909/1836] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:40.603688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.618959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6747bd-c8bf-4823-bda5-1890c4efb0e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.630277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17a8e8-2f9d-49cc-bcf1-e430d145b595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.631607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a8e8-2f9d-49cc-bcf1-e430d145b595 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.632248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a8e8-2f9d-49cc-bcf1-e430d145b595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.632718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a8e8-2f9d-49cc-bcf1-e430d145b595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.633842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17a8e8-2f9d-49cc-bcf1-e430d145b595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.636758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17a8e8-2f9d-49cc-bcf1-e430d145b595 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.654360 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 928/1837] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:40.658364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:40.671931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bb780-41af-463b-a0e1-6173c4316977 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:40.681044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8e1e1-6260-497f-acd7-dd613fc6d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:40.682170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e1e1-6260-497f-acd7-dd613fc6d93e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.682558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e1e1-6260-497f-acd7-dd613fc6d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.682932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e1e1-6260-497f-acd7-dd613fc6d93e admin admin] PATH_INFO: `/v3/projects/20f5c55536fd47d3b0422e693287d341/users/6ba368d37850439eb5683fdf5be0bf76/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.728431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8e1e1-6260-497f-acd7-dd613fc6d93e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f5c55536fd47d3b0422e693287d341, user_id=6ba368d37850439eb5683fdf5be0bf76, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:40.732281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8e1e1-6260-497f-acd7-dd613fc6d93e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:40.782647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 910/1838] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:40 2022] PUT /identity/v3/projects/20f5c55536fd47d3b0422e693287d341/users/6ba368d37850439eb5683fdf5be0bf76/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:40.784199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.784732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.785219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.818212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a73344-ba8a-429b-a597-bbdba215379c None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.818542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a73344-ba8a-429b-a597-bbdba215379c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.820589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a73344-ba8a-429b-a597-bbdba215379c None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.822499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 911/1839] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:43:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:40.826755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] MFA Rules not processed for user `6ba368d37850439eb5683fdf5be0bf76`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:40.833590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:40.833884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:40.834000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:40.834371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:40.871222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:40.873855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:41.017957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 929/1840] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:41.025758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.047694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 912/1841] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:41.074506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.077033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:41.077860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:41.094688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c828691-0c59-42c4-86d0-ff7e21e032db None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.113184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.113910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.114475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.114475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.139908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.141521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.158990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 913/1842] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:41.174741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4521a73-783f-44dd-aa81-062714f169d7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.182466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.183227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.183518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.183798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.184506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.187237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.222090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 930/1843] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:41 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:41.226178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.239656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9ed4a-eb7e-4d0e-9ebd-f2bec3579b31 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.245543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:41.354380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.355107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.355382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.355822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.356652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.359308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.394247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 914/1844] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:41 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:41.397927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.410560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629041bb-7513-4a52-9b98-f710a646dfc4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.416196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.417274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.417751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.418205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.420144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.421973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.435558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 931/1845] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:41.439708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.462422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6157a59-f07d-4a1d-9a6c-565c07812a5d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.468625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa34ae6-7fe9-438c-9006-580cf99452b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.469334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34ae6-7fe9-438c-9006-580cf99452b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.469650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34ae6-7fe9-438c-9006-580cf99452b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.470682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34ae6-7fe9-438c-9006-580cf99452b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.471447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa34ae6-7fe9-438c-9006-580cf99452b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.473708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa34ae6-7fe9-438c-9006-580cf99452b1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.485694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 915/1846] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:41.490471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.501687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90206a-5b1a-4860-95df-f34b75d263cf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.506992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143caa87-2e16-4b1e-924c-9e15188535c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.507908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa87-2e16-4b1e-924c-9e15188535c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.508198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa87-2e16-4b1e-924c-9e15188535c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.508499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa87-2e16-4b1e-924c-9e15188535c9 admin admin] PATH_INFO: `/v3/projects/e2ecd712d233455ba0ecc4033ebbd88d/users/3bb318b898df40a196276617d19353a3/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.528699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143caa87-2e16-4b1e-924c-9e15188535c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ecd712d233455ba0ecc4033ebbd88d, user_id=3bb318b898df40a196276617d19353a3, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.531958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143caa87-2e16-4b1e-924c-9e15188535c9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.552095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 932/1847] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:41 2022] PUT /identity/v3/projects/e2ecd712d233455ba0ecc4033ebbd88d/users/3bb318b898df40a196276617d19353a3/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:41.563524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.563823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.564792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.572517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0fa8f-7ad7-46f7-b19b-0d2a11bd9ddc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.572813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0fa8f-7ad7-46f7-b19b-0d2a11bd9ddc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.573095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0fa8f-7ad7-46f7-b19b-0d2a11bd9ddc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.574658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 933/1848] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:43:41.582652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.582942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.583226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.602148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] MFA Rules not processed for user `3bb318b898df40a196276617d19353a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:41.603451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:41.606021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:41.608138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:41.695678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 934/1849] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:43:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:41.711523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.713398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:41.720044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 916/1850] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:41.724567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.729501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a540137-2dc1-48fb-be4a-4a38ff307be7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.735556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.736282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.736852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.736852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.746355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.748117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.759279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 935/1851] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:41.800193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f468c-6f7d-4a0e-be62-a238b1680cfc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.806724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.807480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.807842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.808205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.809168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.812180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.838735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:41.851470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 917/1852] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:41 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:41.853133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.864676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a772fdd-5239-4a4c-bd4b-7982cbdcbf48 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.921333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:41.946937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.947900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.948188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.948468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:41.949210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:41.951631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:41.978818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 936/1853] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:41 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:41.980117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:41.992286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d9906-91ca-425c-a78e-c141136b5709 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:41.998084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:41.998685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:41.998977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:41.999264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.000002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:42.001635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:42.012214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 918/1854] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:42.017142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:42.027069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9536-3b25-419c-b851-53786688631b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:42.032153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:42.033039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.033325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.033605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] PATH_INFO: `/v3/projects/6997aaac828a4506891b92b842bd339e/users/75cfef5ef61245eeb1f867c13b6b7b3d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.048227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6997aaac828a4506891b92b842bd339e, user_id=75cfef5ef61245eeb1f867c13b6b7b3d, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:42.050906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:42.064159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 937/1855] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:42 2022] PUT /identity/v3/projects/6997aaac828a4506891b92b842bd339e/users/75cfef5ef61245eeb1f867c13b6b7b3d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:42.068117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:42.079305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b742e36-5110-4ae3-bd08-f8d891d9022d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:42.141621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda923d-3d89-45de-bff4-4a17c3fd26fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:42.142325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda923d-3d89-45de-bff4-4a17c3fd26fc admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.142613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda923d-3d89-45de-bff4-4a17c3fd26fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.142894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda923d-3d89-45de-bff4-4a17c3fd26fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.143625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda923d-3d89-45de-bff4-4a17c3fd26fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:42.145209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda923d-3d89-45de-bff4-4a17c3fd26fc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:42.155636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 919/1856] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:42.159973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:42.173881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0840b-a0ea-4c61-9d67-ac579e070079 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:42.178507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:42.179421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.179707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.180012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] PATH_INFO: `/v3/projects/6997aaac828a4506891b92b842bd339e/users/75cfef5ef61245eeb1f867c13b6b7b3d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.182021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6997aaac828a4506891b92b842bd339e, user_id=75cfef5ef61245eeb1f867c13b6b7b3d, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:42.185140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:42.195984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 938/1857] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:42 2022] PUT /identity/v3/projects/6997aaac828a4506891b92b842bd339e/users/75cfef5ef61245eeb1f867c13b6b7b3d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:42.205963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.206260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.206549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.237823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] MFA Rules not processed for user `75cfef5ef61245eeb1f867c13b6b7b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:42.238959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:42.330397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 920/1858] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:42.343252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:42.349378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:42.411316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9684e8-8984-4b68-9773-e98e0883a80c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:42.414986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:42.415637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:42.415918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:42.416194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:42.477757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:42.479190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:42.488311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 939/1859] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:50.391366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:50.392266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:50.407278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac043d3-b041-4435-b411-d63cfb33e413 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:50.411268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:50.411701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:50.411986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:50.412263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:50.485403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:50.486800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:50.499722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 921/1860] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:51.239018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:51.250662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346d133-189a-4108-8601-aada76e401f3 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:51.256077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677e794-1fec-4743-9a99-695fc6bfa29f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:51.257066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677e794-1fec-4743-9a99-695fc6bfa29f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:51.257486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677e794-1fec-4743-9a99-695fc6bfa29f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:51.258033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677e794-1fec-4743-9a99-695fc6bfa29f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:51.269216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677e794-1fec-4743-9a99-695fc6bfa29f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:51.270895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677e794-1fec-4743-9a99-695fc6bfa29f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:51.303335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 940/1861] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:51.690512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:51.692088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:51.704937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:51.767216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8b57a-0fa0-4449-a977-f9168635659b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:51.771862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:51.772683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:51.772975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:51.773259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] PATH_INFO: `/v3/users/8947d5e075e345029706aa45253e055a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:51.774494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8947d5e075e345029706aa45253e055a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:51.776406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:51.820977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8947d5e075e345029706aa45253e055a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:51.833294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8947d5e075e345029706aa45253e055a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:51.840560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8947d5e075e345029706aa45253e055a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:51.848886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8947d5e075e345029706aa45253e055a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:51.849636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8947d5e075e345029706aa45253e055a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:51.855605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 922/1862] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:51 2022] DELETE /identity/v3/users/8947d5e075e345029706aa45253e055a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:51.860958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:51.861257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:51.861537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:51.878392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:51.879614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:51.917306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:51.940539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 941/1863] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:51.950272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:51.951873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:52.010350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50913a3e-2428-4dbf-b11b-77ea44c378dd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:52.014810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:52.015470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:52.015760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:52.016043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:52.025508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:52.026893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:52.037401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 923/1864] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:52.338039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:52.339584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:52.352672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75266ecb-0e94-4761-9534-7afa020a38b5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:52.357549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:52.358470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:52.358766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:52.359049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] PATH_INFO: `/v3/projects/3e81cc8aeeff4d299e5c3c51ab0c138b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:52.360380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e81cc8aeeff4d299e5c3c51ab0c138b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:52.362218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:52.409280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e81cc8aeeff4d299e5c3c51ab0c138b', 'request_id': 'req-942bf223-cdf4-489a-b431-b0d41dbab4d3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.416591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e81cc8aeeff4d299e5c3c51ab0c138b', 'request_id': 'req-942bf223-cdf4-489a-b431-b0d41dbab4d3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.422181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e81cc8aeeff4d299e5c3c51ab0c138b', 'request_id': 'req-942bf223-cdf4-489a-b431-b0d41dbab4d3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.432806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] The token cache is being invalidate because project 3e81cc8aeeff4d299e5c3c51ab0c138b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:52.433094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.433757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.435865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 942/1865] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:52 2022] DELETE /identity/v3/projects/3e81cc8aeeff4d299e5c3c51ab0c138b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:52.440048 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:52.449789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e042f-72e9-421b-9204-32015a2a87e3 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:52.479448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:52.509433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:52.510352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:52.510639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:52.510923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] PATH_INFO: `/v3/users/c59a2d16c76144899139efc64dc3d1bc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:52.512134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59a2d16c76144899139efc64dc3d1bc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:52.514869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:52.574475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59a2d16c76144899139efc64dc3d1bc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.587558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59a2d16c76144899139efc64dc3d1bc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.594964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59a2d16c76144899139efc64dc3d1bc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.603359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59a2d16c76144899139efc64dc3d1bc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.603976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fda625e6-8504-4c6d-812e-a89e038f345d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59a2d16c76144899139efc64dc3d1bc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:52.611488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 924/1866] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:52 2022] DELETE /identity/v3/users/c59a2d16c76144899139efc64dc3d1bc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:53.042359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.043787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:53.058621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bf223-cdf4-489a-b431-b0d41dbab4d3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.149959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.151568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.152141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.152774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] PATH_INFO: `/v3/projects/1b78473b6c30427fb258c866c9cfb0d1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.155616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b78473b6c30427fb258c866c9cfb0d1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.159995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.247052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b78473b6c30427fb258c866c9cfb0d1', 'request_id': 'req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.258873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b78473b6c30427fb258c866c9cfb0d1', 'request_id': 'req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.269290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b78473b6c30427fb258c866c9cfb0d1', 'request_id': 'req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.273116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:53.288155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] The token cache is being invalidate because project 1b78473b6c30427fb258c866c9cfb0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:53.289040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.290271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-790c4def-1c22-4f3d-82d0-4ab7ca8cb000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:53.292571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 943/1867] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:53 2022] DELETE /identity/v3/projects/1b78473b6c30427fb258c866c9cfb0d1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:53.297848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.298343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.298454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.318553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:53.320150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:53.396149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 925/1868] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:53.403575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.404128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.404128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.424594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:53.425829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:53.433655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 944/1869] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:53.438636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.448534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd3cb6-8bf1-418c-bc21-d7e2502b023a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.453991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.454685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.454974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.455284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.456024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:53.456868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.457959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.470261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 926/1870] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:53.474621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.485120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a170db1-352e-4e2c-ac44-d4b6a3e09fb2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.489821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.490477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.490768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.491050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.491750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.493653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.519325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 945/1871] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:53.523462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.533317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926120-5159-4ad1-9107-bade0a7ed46b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.566806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:53.594690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.595388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.595681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.595967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.596687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.598938 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.624283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 927/1872] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:53.627894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.647640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cdcdd-4ffd-4d38-b74a-7afad05cc721 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.652365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.653137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.653492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.653873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.655039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.656798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.667384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 946/1873] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:53.673374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.684300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c00aad-6652-48c3-8d62-e515dcb359eb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.690096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994664f-e617-4ef9-af7f-6953bd646b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.690771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994664f-e617-4ef9-af7f-6953bd646b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.691209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994664f-e617-4ef9-af7f-6953bd646b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.691508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994664f-e617-4ef9-af7f-6953bd646b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.692244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994664f-e617-4ef9-af7f-6953bd646b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.693848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994664f-e617-4ef9-af7f-6953bd646b5b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.704894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 928/1874] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:53.709096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.723362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c0a2c-0951-461c-a853-15a08cce031b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.729377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.730326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.730613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.730891 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] PATH_INFO: `/v3/projects/bd5e254c07fd4ad89eda3dbc65a14688/users/a6fe03ede6704515b7b572a988a152d2/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.746003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5e254c07fd4ad89eda3dbc65a14688, user_id=a6fe03ede6704515b7b572a988a152d2, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.748972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.762301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 947/1875] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:53 2022] PUT /identity/v3/projects/bd5e254c07fd4ad89eda3dbc65a14688/users/a6fe03ede6704515b7b572a988a152d2/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:53.774035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.774402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.774705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.801895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] MFA Rules not processed for user `a6fe03ede6704515b7b572a988a152d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:53.803125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:53.870405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 929/1876] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:53.875488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:53.949026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb67910-cdd1-4394-b27c-dc180aedf45e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:53.953916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:53.954585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:53.954873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:53.955151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:53.955862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:53.957774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:53.988027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 948/1877] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:53.989160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.001113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657ba49-a27f-4ce2-8796-93292e858d3e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.071761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.072650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.073036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.073950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.074794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.077883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.114304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 930/1878] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:53 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:54.118033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.140304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a9e25-81cf-4b29-aabd-cccd48801fbb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.148372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.149059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.149347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.149631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.150396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.152024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.164075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 949/1879] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:54.168372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.179767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b60cb7-0817-4a3c-b9bb-0d4d7bc40428 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.184514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.185973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.187338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.187338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] PATH_INFO: `/v3/projects/97ed4b1733234cb081211621e6f1c537/users/60ae2af0d0184aa081087b8a915e632f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.203677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ed4b1733234cb081211621e6f1c537, user_id=60ae2af0d0184aa081087b8a915e632f, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.206476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.220079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 931/1880] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:54 2022] PUT /identity/v3/projects/97ed4b1733234cb081211621e6f1c537/users/60ae2af0d0184aa081087b8a915e632f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:54.224047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.234938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b9c2c-14df-434f-814e-c14bcde8a1ba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.293684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf5bca-0b9a-4993-ab6c-e50c8f91b91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.294362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5bca-0b9a-4993-ab6c-e50c8f91b91d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.294680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5bca-0b9a-4993-ab6c-e50c8f91b91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.294978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5bca-0b9a-4993-ab6c-e50c8f91b91d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.295799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5bca-0b9a-4993-ab6c-e50c8f91b91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.297393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5bca-0b9a-4993-ab6c-e50c8f91b91d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.307094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 950/1881] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:54.311627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.322500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91fd09-20a4-4c38-8283-f2752dcfbbb7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.328176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.329102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.329726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.329984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] PATH_INFO: `/v3/projects/97ed4b1733234cb081211621e6f1c537/users/60ae2af0d0184aa081087b8a915e632f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.332176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ed4b1733234cb081211621e6f1c537, user_id=60ae2af0d0184aa081087b8a915e632f, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.334920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.340885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:54.347843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 932/1882] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:54 2022] PUT /identity/v3/projects/97ed4b1733234cb081211621e6f1c537/users/60ae2af0d0184aa081087b8a915e632f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:54.358099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.358395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.358679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.364419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:54.387369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] MFA Rules not processed for user `60ae2af0d0184aa081087b8a915e632f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:54.389428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:54.462700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 951/1883] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:54.474572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:54.476480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:54.537128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee3f72-72ff-49fa-96ee-a261c039c46f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:54.540819 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725d1c3-1a41-44dd-a899-faa4a32ecc5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:54.541482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725d1c3-1a41-44dd-a899-faa4a32ecc5e service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:54.541794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725d1c3-1a41-44dd-a899-faa4a32ecc5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:54.542082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725d1c3-1a41-44dd-a899-faa4a32ecc5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:54.551310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725d1c3-1a41-44dd-a899-faa4a32ecc5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:54.552731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725d1c3-1a41-44dd-a899-faa4a32ecc5e service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:54.562207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 933/1884] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:54.994188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.061046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf744d9-5cde-43dc-8eb0-d267e257d616 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.066548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.067443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.067723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.067996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] PATH_INFO: `/v3/users/a6fe03ede6704515b7b572a988a152d2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.069199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6fe03ede6704515b7b572a988a152d2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.071200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.125086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6fe03ede6704515b7b572a988a152d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.125785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6fe03ede6704515b7b572a988a152d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.132847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6fe03ede6704515b7b572a988a152d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.133426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6fe03ede6704515b7b572a988a152d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.142689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6fe03ede6704515b7b572a988a152d2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.163549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 952/1885] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:54 2022] DELETE /identity/v3/users/a6fe03ede6704515b7b572a988a152d2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:55.168014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.168014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.168283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.187731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:55.189089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:55.277220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 934/1886] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:55.291729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.293136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:55.313743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:55.375798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1273c26-5289-40fd-af5f-ce326833ed7a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.380426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.381345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.381850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.382337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.393216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.394796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.407744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 953/1887] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:55.696890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.698375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:55.713810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cf890-2ac9-4e11-a5b0-da20351fe267 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.720550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.721629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.722139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.722735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] PATH_INFO: `/v3/projects/bd5e254c07fd4ad89eda3dbc65a14688` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.724662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5e254c07fd4ad89eda3dbc65a14688)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.727361 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:55.786318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5e254c07fd4ad89eda3dbc65a14688', 'request_id': 'req-7b943cf5-7464-49d0-a201-55af3cbcac12'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.793129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5e254c07fd4ad89eda3dbc65a14688', 'request_id': 'req-7b943cf5-7464-49d0-a201-55af3cbcac12'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.801714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5e254c07fd4ad89eda3dbc65a14688', 'request_id': 'req-7b943cf5-7464-49d0-a201-55af3cbcac12'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.819784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] The token cache is being invalidate because project bd5e254c07fd4ad89eda3dbc65a14688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:55.820276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.821109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:55.823039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 935/1888] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:55 2022] DELETE /identity/v3/projects/bd5e254c07fd4ad89eda3dbc65a14688 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:55.827449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:55.839818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa2d5-19b8-4045-b7da-bd2ff6a3ab91 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:55.925956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:55.928196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:55.928775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:55.929199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] PATH_INFO: `/v3/users/60ae2af0d0184aa081087b8a915e632f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:55.931424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ae2af0d0184aa081087b8a915e632f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:55.933815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.061775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ae2af0d0184aa081087b8a915e632f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.062586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ae2af0d0184aa081087b8a915e632f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.072643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ae2af0d0184aa081087b8a915e632f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.075210 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ae2af0d0184aa081087b8a915e632f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.088984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2060033a-aa4d-44c1-964b-29bef6464ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ae2af0d0184aa081087b8a915e632f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.137071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 954/1889] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:55 2022] DELETE /identity/v3/users/60ae2af0d0184aa081087b8a915e632f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:56.545267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:56.547031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:56.570725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b943cf5-7464-49d0-a201-55af3cbcac12 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:56.613595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:56.680210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:56.681118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.681442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.681708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] PATH_INFO: `/v3/projects/97ed4b1733234cb081211621e6f1c537` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.683127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ed4b1733234cb081211621e6f1c537)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:56.685119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:56.755443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ed4b1733234cb081211621e6f1c537', 'request_id': 'req-26fa8e18-3893-4527-9ea3-594a322ef6b1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.762965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ed4b1733234cb081211621e6f1c537', 'request_id': 'req-26fa8e18-3893-4527-9ea3-594a322ef6b1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.771046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ed4b1733234cb081211621e6f1c537', 'request_id': 'req-26fa8e18-3893-4527-9ea3-594a322ef6b1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.793712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] The token cache is being invalidate because project 97ed4b1733234cb081211621e6f1c537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:56.794031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.795287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-26fa8e18-3893-4527-9ea3-594a322ef6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:56.797336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 936/1890] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:56 2022] DELETE /identity/v3/projects/97ed4b1733234cb081211621e6f1c537 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:56.802704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:56.803082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:56.803434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:56.825928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:56.827981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:56.893120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:56.997741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 955/1891] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:57.008149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.008913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.008913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.038789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:57.042261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.061530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 937/1892] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:57.062011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.074963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27620a2c-5c5a-4bb0-b1a6-5087c0109c6a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.090570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.091321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.091718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.092603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.093697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:57.093823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.097494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.117277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 956/1893] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:57.121731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.133140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2e994-879f-4b4a-b040-dea83e55dcd3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.139568 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.140412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.141009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.141532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.142566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.144564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.182090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 938/1894] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:57.186726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.198191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a934bbd-8a45-4311-8855-e085edba866f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.272888 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.273631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.273952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.274230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.274942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.277004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.303728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 957/1895] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:57.307634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.326930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc925b4-0328-4b6d-884d-899c2224539f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.334120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.335046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.335538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.336165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.337003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.338735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.352147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 939/1896] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:57.355904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.368099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64165193-fde0-4ce4-ae83-fc2693a6dd64 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.373497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.374613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.374914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.375200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.375940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.377529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.392559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 958/1897] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:57.393675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.411786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9ca4-70e0-467e-b206-f03a24ec6400 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.417410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dbdec-f5ad-4d23-b8e9-b563750af96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.418404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dbdec-f5ad-4d23-b8e9-b563750af96e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.418769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dbdec-f5ad-4d23-b8e9-b563750af96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.419125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dbdec-f5ad-4d23-b8e9-b563750af96e admin admin] PATH_INFO: `/v3/projects/8dedcef64d17482092ccbec70b1d482c/users/801dcae783b44b3aa029057ed4e8e90a/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.429290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.431230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.453703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bd2e-b8d0-4533-aaac-e0541431ab13 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.461782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dbdec-f5ad-4d23-b8e9-b563750af96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dedcef64d17482092ccbec70b1d482c, user_id=801dcae783b44b3aa029057ed4e8e90a, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.462227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.463253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.464046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.464046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] PATH_INFO: `/v3/users/441bd4602a42415eb332b791d4aa7d69` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.465368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=441bd4602a42415eb332b791d4aa7d69)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.465890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dbdec-f5ad-4d23-b8e9-b563750af96e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.468418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.495657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 940/1898] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:57 2022] PUT /identity/v3/projects/8dedcef64d17482092ccbec70b1d482c/users/801dcae783b44b3aa029057ed4e8e90a/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:57.497213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.497604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.498026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.531209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] MFA Rules not processed for user `801dcae783b44b3aa029057ed4e8e90a`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:57.532282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.554034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441bd4602a42415eb332b791d4aa7d69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.557991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441bd4602a42415eb332b791d4aa7d69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.570395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441bd4602a42415eb332b791d4aa7d69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.571234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441bd4602a42415eb332b791d4aa7d69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.581118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21a07e02-fe8a-4ab2-8e36-673b66af03cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441bd4602a42415eb332b791d4aa7d69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:57.586851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:57.616773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 959/1899] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:57 2022] DELETE /identity/v3/users/441bd4602a42415eb332b791d4aa7d69 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:57.619175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 941/1900] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:57.621306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.622123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.622123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.624827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.626402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.641584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:57.643464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:57.645392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.698581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b7f12-1de5-4a99-8f16-c6d5504e0b8e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.709574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 960/1901] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:57.720357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.722042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:57.765944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.766652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.766918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.767207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.767919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.769984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.789654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25430-6bea-40e3-bdb5-d63128777226 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.793879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.794513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.795486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.795762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.805901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 942/1902] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:57.807323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.808916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.811659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.823215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 961/1903] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:57.881580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260b72-a843-4305-a404-561d55886357 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.886914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.887973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.888263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.889067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.890306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.892596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.921689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 943/1904] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:57 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:57.923528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.933929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe328ad-7005-4b56-9676-5dd007e39ebb service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.938778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.939390 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.939671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.939955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.940699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.942338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.953951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 962/1905] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:57.957941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:57.963376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:57.971083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d979-1907-48d1-929f-9e5477a51639 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:57.976241 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c1064-0362-44af-962c-c2820b193e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:57.976979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c1064-0362-44af-962c-c2820b193e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:57.977275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c1064-0362-44af-962c-c2820b193e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:57.977569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c1064-0362-44af-962c-c2820b193e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:57.978682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c1064-0362-44af-962c-c2820b193e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:57.979931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c1064-0362-44af-962c-c2820b193e37 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:57.991257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 944/1906] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:57.995334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.009456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f117d-d1b5-4cae-84a5-6f14bc6cb43e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.014621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.015520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.015812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.016092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] PATH_INFO: `/v3/projects/68a54047a722433f9c8c84bfc4360a74/users/95c676a7004c403b92ab4f769aa7663e/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.032606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a54047a722433f9c8c84bfc4360a74, user_id=95c676a7004c403b92ab4f769aa7663e, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.035387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.049851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 963/1907] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:57 2022] PUT /identity/v3/projects/68a54047a722433f9c8c84bfc4360a74/users/95c676a7004c403b92ab4f769aa7663e/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:58.059894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.060224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.060565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.091507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] MFA Rules not processed for user `95c676a7004c403b92ab4f769aa7663e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:58.093597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.146921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.149034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.187581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 945/1908] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:58.191581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.245399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3b8cc-7c4b-4f61-acde-1f77952c1da2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.251697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.252575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.252866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.253152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] PATH_INFO: `/v3/projects/da9c99432ab44a84ba98417ae23ce392` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.256149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9c99432ab44a84ba98417ae23ce392)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.258357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.273691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89dd23-7184-46da-8c62-b6101d0f4f52 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.279372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.280323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.280778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.281070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.281182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:58.281873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.284169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.314011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 946/1909] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:58 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:58.317749 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.324677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9c99432ab44a84ba98417ae23ce392', 'request_id': 'req-f5edf279-1f36-439c-8b1e-20da2c52f231'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.333214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9c99432ab44a84ba98417ae23ce392', 'request_id': 'req-f5edf279-1f36-439c-8b1e-20da2c52f231'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.339441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9c99432ab44a84ba98417ae23ce392', 'request_id': 'req-f5edf279-1f36-439c-8b1e-20da2c52f231'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.351191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] The token cache is being invalidate because project da9c99432ab44a84ba98417ae23ce392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:58.351492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.352192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.353802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 964/1910] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:58 2022] DELETE /identity/v3/projects/da9c99432ab44a84ba98417ae23ce392 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:58.357728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.359178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.385258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9683c-e4fd-4cea-b94f-4484aa425a7d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.391058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.391736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.392020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.392303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.392993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.394929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.405492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:58.418217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 947/1911] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:58 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:43:58.422680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.424401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.427913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edf279-1f36-439c-8b1e-20da2c52f231 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.432776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.433797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.434121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.434403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] PATH_INFO: `/v3/users/754fe863dac243e7a99e11aa6de0b722` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.436200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754fe863dac243e7a99e11aa6de0b722)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d346b5-d1fb-459b-b4d0-a642d193d38e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.438167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.441830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.442425 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.442712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.442991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.443724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.445301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.456137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 948/1912] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:58.459978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.470599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201edfb0-063a-4610-a787-c1d1ffad9d07 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.475807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.476700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.476993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.477276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] PATH_INFO: `/v3/projects/d3ecc7b433734aea8b9377d4d37f8844/users/41a9f67bc5aa4480a76f71bb6254b284/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.489513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754fe863dac243e7a99e11aa6de0b722'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.490230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754fe863dac243e7a99e11aa6de0b722'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.493046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ecc7b433734aea8b9377d4d37f8844, user_id=41a9f67bc5aa4480a76f71bb6254b284, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.495775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.497203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754fe863dac243e7a99e11aa6de0b722'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.498574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754fe863dac243e7a99e11aa6de0b722'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.506761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754fe863dac243e7a99e11aa6de0b722'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:58.510174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 949/1913] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:58 2022] PUT /identity/v3/projects/d3ecc7b433734aea8b9377d4d37f8844/users/41a9f67bc5aa4480a76f71bb6254b284/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:58.513978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.515622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.524543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 965/1914] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:43:58 2022] DELETE /identity/v3/users/754fe863dac243e7a99e11aa6de0b722 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:58.581968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e082-4a62-4f55-b471-9bdc0a0da828 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.648485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a816f3d-f6b4-448a-95a5-1025f30efdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.649171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a816f3d-f6b4-448a-95a5-1025f30efdda admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.650102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a816f3d-f6b4-448a-95a5-1025f30efdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.650411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a816f3d-f6b4-448a-95a5-1025f30efdda admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.651153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a816f3d-f6b4-448a-95a5-1025f30efdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.653259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a816f3d-f6b4-448a-95a5-1025f30efdda admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.667265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 950/1915] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:58.668431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.679943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19166-655e-4e0e-8dde-0c57272121f4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.685346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.686440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.686729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.687007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] PATH_INFO: `/v3/projects/d3ecc7b433734aea8b9377d4d37f8844/users/41a9f67bc5aa4480a76f71bb6254b284/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.701111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ecc7b433734aea8b9377d4d37f8844, user_id=41a9f67bc5aa4480a76f71bb6254b284, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.705513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.723308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 966/1916] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:58 2022] PUT /identity/v3/projects/d3ecc7b433734aea8b9377d4d37f8844/users/41a9f67bc5aa4480a76f71bb6254b284/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:58.735368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.735669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.735958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.764926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] MFA Rules not processed for user `41a9f67bc5aa4480a76f71bb6254b284`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:58.766140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.831135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:58.836585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.839036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.862014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 951/1917] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:58.874059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:58.875589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:58.878678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:58.941872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bf3be-67fe-4000-b9c5-269dae902d06 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.942514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee863d0b-ec52-4cba-9cda-a48d51441c79 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:58.946261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.946855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.947104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:58.947226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.947401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.948344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:58.948743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:58.949090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] PATH_INFO: `/v3/projects/ce80155917b04aa3b76ea4c87ff364a3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:58.950485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce80155917b04aa3b76ea4c87ff364a3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.953386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.959503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:58.961626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:58.974883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 952/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:43:59.003444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce80155917b04aa3b76ea4c87ff364a3', 'request_id': 'req-1a477370-1a84-4771-a047-aa8ccc700dca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.018482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce80155917b04aa3b76ea4c87ff364a3', 'request_id': 'req-1a477370-1a84-4771-a047-aa8ccc700dca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.021505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.023051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.035028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e76ee1-6f48-4f33-8ad8-6bf9445be1b9 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.036979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce80155917b04aa3b76ea4c87ff364a3', 'request_id': 'req-1a477370-1a84-4771-a047-aa8ccc700dca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.049103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] The token cache is being invalidate because project ce80155917b04aa3b76ea4c87ff364a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:43:59.049380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.050258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a477370-1a84-4771-a047-aa8ccc700dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:43:59.051900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 967/1919] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:43:58 2022] DELETE /identity/v3/projects/ce80155917b04aa3b76ea4c87ff364a3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:43:59.056632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d09c9-5547-4cee-899c-303cebb37de2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.057000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d09c9-5547-4cee-899c-303cebb37de2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.057359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d09c9-5547-4cee-899c-303cebb37de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.078220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b12d09c9-5547-4cee-899c-303cebb37de2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:59.079525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b12d09c9-5547-4cee-899c-303cebb37de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.093208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.093884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.094171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.094450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.096291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.140474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 968/1920] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:59.148133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.148434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.148724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.158690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.160171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.166247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:59.167452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.169408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 953/1921] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:43:59.176544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 969/1922] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:43:59.180527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.192009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cf5ea-e765-41f7-8989-a634b6e90127 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.196913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.197577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.198034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.198387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.199416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:43:59.199834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.201732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.221838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 954/1923] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:43:59.223787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.235241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980ccf2-bdef-4f8e-b0b2-fd18a60ff370 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.241508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.242243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.242535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.242807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.243510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.245464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.280250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 970/1924] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:59.285112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.296702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1f6ed-d1cd-41f7-8563-9e5e75992f3a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.336230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:59.367092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.367905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.368294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.368680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.369630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.371738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.401709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 955/1925] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:59.406079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.418929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22526c-a8fc-4908-9c87-d59c138969b9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.425004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.425718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.426009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.426284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.427025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.428618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.443103 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 971/1926] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:59.447596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.463381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e9cc1-9e72-43f6-ae87-0b5772b6e856 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.473273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4025e-d457-44a8-89dc-feb96bacd730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.474368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4025e-d457-44a8-89dc-feb96bacd730 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.474912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4025e-d457-44a8-89dc-feb96bacd730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.475141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4025e-d457-44a8-89dc-feb96bacd730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.475880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4025e-d457-44a8-89dc-feb96bacd730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.477615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4025e-d457-44a8-89dc-feb96bacd730 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.491979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 956/1927] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:43:59.496447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.503430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:43:59.510552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd46b-4696-4c4f-beb4-f83376fcd8c1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.517639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.518963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.519211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.519475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] PATH_INFO: `/v3/projects/f487c890763249b8afd4cd1f9062010a/users/51a3f540db7a4134a6ba120819747b90/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.539827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f487c890763249b8afd4cd1f9062010a, user_id=51a3f540db7a4134a6ba120819747b90, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.543034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.591152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 972/1928] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:59 2022] PUT /identity/v3/projects/f487c890763249b8afd4cd1f9062010a/users/51a3f540db7a4134a6ba120819747b90/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:43:59.594618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.594937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.595291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.627807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] MFA Rules not processed for user `51a3f540db7a4134a6ba120819747b90`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:43:59.629037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:43:59.695456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 957/1929] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:43:59.699864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.762822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a05325-6e08-4bf6-9d29-bcf37e040245 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.768086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.768746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.769037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.769316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.770069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.773363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.804222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 973/1930] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:59.808355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.819791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579cc8-ac25-4e77-a5e5-169cc5cbbaa3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.883303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.883997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.884426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.884715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.885699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.887366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.914132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 958/1931] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:43:59 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:43:59.915325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.926440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca638-8c3a-43bb-b833-d8940e81ed68 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.931173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.931848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.932142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.932423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.933158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.934786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:43:59.945941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 974/1932] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:43:59.950152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:43:59.961194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbeded-bd2b-4ea8-b779-ddd1c33bd68f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:43:59.965936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:43:59.967517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:43:59.967826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:43:59.968103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] PATH_INFO: `/v3/projects/3ed1e2ad104545e7bad3960f38b77b85/users/35b091d8b8334fcea70572fb95f8bca5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:43:59.985559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed1e2ad104545e7bad3960f38b77b85, user_id=35b091d8b8334fcea70572fb95f8bca5, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:43:59.988349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.004864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 959/1933] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:43:59 2022] PUT /identity/v3/projects/3ed1e2ad104545e7bad3960f38b77b85/users/35b091d8b8334fcea70572fb95f8bca5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:00.008830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.019322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d40da-ad21-4f1e-a186-b226203196fe admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.074431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a7e62-fc0b-4646-b176-23d73ec85031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.075097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a7e62-fc0b-4646-b176-23d73ec85031 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.075380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a7e62-fc0b-4646-b176-23d73ec85031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.075658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a7e62-fc0b-4646-b176-23d73ec85031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.076472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a7e62-fc0b-4646-b176-23d73ec85031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.078098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a7e62-fc0b-4646-b176-23d73ec85031 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.087966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 975/1934] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:00.091966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.102739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c842a-845d-4e68-b196-e90e21f0f50f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.108291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.109198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.109489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.109808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] PATH_INFO: `/v3/projects/3ed1e2ad104545e7bad3960f38b77b85/users/35b091d8b8334fcea70572fb95f8bca5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.111863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed1e2ad104545e7bad3960f38b77b85, user_id=35b091d8b8334fcea70572fb95f8bca5, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.114618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.117311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:00.129075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 960/1935] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:00 2022] PUT /identity/v3/projects/3ed1e2ad104545e7bad3960f38b77b85/users/35b091d8b8334fcea70572fb95f8bca5/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:00.140285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.140608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.140894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.167182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] MFA Rules not processed for user `35b091d8b8334fcea70572fb95f8bca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:00.168398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:00.207902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:00.241379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 976/1936] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:00.252991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.254731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:00.324147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613260c-bbfe-49a1-80cb-9e54a0f06a4d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.329050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.329836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.330199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.330553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.340770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.342759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.352841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 961/1937] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:00.397951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:00.399362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:00.412029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cde5f-86cd-4121-8f93-45b3e4984b5d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:00.415716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd702f-d317-437a-a2a3-f335441043f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:00.416388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd702f-d317-437a-a2a3-f335441043f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:00.416677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd702f-d317-437a-a2a3-f335441043f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:00.416960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd702f-d317-437a-a2a3-f335441043f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:00.481188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd702f-d317-437a-a2a3-f335441043f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:00.483449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd702f-d317-437a-a2a3-f335441043f0 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:00.492999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 977/1938] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:02.932845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:02.934398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.017155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084f4b6-1ac8-46d2-ab7f-573ce4fc7f39 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.022861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.024515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.024972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.025438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] PATH_INFO: `/v3/users/6ba368d37850439eb5683fdf5be0bf76` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.027050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba368d37850439eb5683fdf5be0bf76)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.029013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.082552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba368d37850439eb5683fdf5be0bf76'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.099662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba368d37850439eb5683fdf5be0bf76'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.106411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba368d37850439eb5683fdf5be0bf76'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.116559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba368d37850439eb5683fdf5be0bf76'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.117336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba368d37850439eb5683fdf5be0bf76'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.124507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 962/1939] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:02 2022] DELETE /identity/v3/users/6ba368d37850439eb5683fdf5be0bf76 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.129007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.129480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.129568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.146231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.147447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.205949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 978/1940] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.215757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.217262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.235940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:03.277209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971eef-f08b-44a3-a50d-96b2518b0fed admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.282406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068a1f-890c-49f9-a0f9-ef4d67031731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.283195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068a1f-890c-49f9-a0f9-ef4d67031731 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.283409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068a1f-890c-49f9-a0f9-ef4d67031731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.283625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068a1f-890c-49f9-a0f9-ef4d67031731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.293220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4068a1f-890c-49f9-a0f9-ef4d67031731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.294818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4068a1f-890c-49f9-a0f9-ef4d67031731 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.306014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 963/1941] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:03.411048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.413707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.432596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77929b74-9240-4ee4-b220-9b6b5d69a836 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.438623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.439622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.439972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.440476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] PATH_INFO: `/v3/users/801dcae783b44b3aa029057ed4e8e90a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.442189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] RBAC: Authorizing `identity:delete_user(user_id=801dcae783b44b3aa029057ed4e8e90a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.444973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.543288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801dcae783b44b3aa029057ed4e8e90a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.544093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801dcae783b44b3aa029057ed4e8e90a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.554555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801dcae783b44b3aa029057ed4e8e90a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.556148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801dcae783b44b3aa029057ed4e8e90a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.574178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801dcae783b44b3aa029057ed4e8e90a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.586572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:03.650491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 979/1942] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/users/801dcae783b44b3aa029057ed4e8e90a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.652549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.653007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.653439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.676510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:03.677360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.750400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 964/1943] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:03.757451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.759770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.761106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.764573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:03.778739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5179c-781e-4b13-9a40-2f916e31845d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.785573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.786654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.786946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.787226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] PATH_INFO: `/v3/projects/20f5c55536fd47d3b0422e693287d341` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.789150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f5c55536fd47d3b0422e693287d341)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.791595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.856315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa923-5c2d-424b-9142-4d1c202f9a9a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:03.864605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:03.865276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:03.865566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:03.865879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:03.876334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f5c55536fd47d3b0422e693287d341', 'request_id': 'req-414debc7-7f6c-4985-b49b-b6c282dc5066'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.876656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:03.878158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:03.889246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f5c55536fd47d3b0422e693287d341', 'request_id': 'req-414debc7-7f6c-4985-b49b-b6c282dc5066'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.890059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 965/1944] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:03.900076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f5c55536fd47d3b0422e693287d341', 'request_id': 'req-414debc7-7f6c-4985-b49b-b6c282dc5066'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.913508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] The token cache is being invalidate because project 20f5c55536fd47d3b0422e693287d341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:03.913918 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.914673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:03.916814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 980/1945] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/projects/20f5c55536fd47d3b0422e693287d341 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:03.922642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:03.934506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803458b-bc36-434a-ac46-94afb8a70705 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.026286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.027227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.027507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.027811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] PATH_INFO: `/v3/users/3bb318b898df40a196276617d19353a3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.030620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb318b898df40a196276617d19353a3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.033297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.036328 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.103033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb318b898df40a196276617d19353a3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.126390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb318b898df40a196276617d19353a3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.136640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb318b898df40a196276617d19353a3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.149269 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb318b898df40a196276617d19353a3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.150287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb318b898df40a196276617d19353a3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.158557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 966/1946] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:03 2022] DELETE /identity/v3/users/3bb318b898df40a196276617d19353a3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.237594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.239292 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.253436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414debc7-7f6c-4985-b49b-b6c282dc5066 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.329195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.330223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.330511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.330802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.331081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] PATH_INFO: `/v3/users/51a3f540db7a4134a6ba120819747b90` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.332257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.332723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a3f540db7a4134a6ba120819747b90)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.335383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.346446 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e6a9-677e-4e69-8242-7ecc9598fb54 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.352365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.353896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.354253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.354622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] PATH_INFO: `/v3/projects/8dedcef64d17482092ccbec70b1d482c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.356134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dedcef64d17482092ccbec70b1d482c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.359164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.407536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a3f540db7a4134a6ba120819747b90'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.408233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a3f540db7a4134a6ba120819747b90'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.417350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a3f540db7a4134a6ba120819747b90'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.418440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a3f540db7a4134a6ba120819747b90'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.430482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3716a619-1e5a-4284-86b8-125ea4886590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a3f540db7a4134a6ba120819747b90'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.433227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dedcef64d17482092ccbec70b1d482c', 'request_id': 'req-4a0cb76e-7db6-4679-81f3-9dfd56b863be'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.440232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.440875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dedcef64d17482092ccbec70b1d482c', 'request_id': 'req-4a0cb76e-7db6-4679-81f3-9dfd56b863be'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.454109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dedcef64d17482092ccbec70b1d482c', 'request_id': 'req-4a0cb76e-7db6-4679-81f3-9dfd56b863be'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.465272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 981/1947] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:04 2022] DELETE /identity/v3/users/51a3f540db7a4134a6ba120819747b90 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.470594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.470880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.471352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.473695 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] The token cache is being invalidate because project 8dedcef64d17482092ccbec70b1d482c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:04.473999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.474668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.476449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 967/1948] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:04 2022] DELETE /identity/v3/projects/8dedcef64d17482092ccbec70b1d482c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.480121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.481923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.492463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:04.494121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.567328 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cb76e-7db6-4679-81f3-9dfd56b863be admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.577824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.578823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 982/1949] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:04.580064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.581690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.594849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cb557-f26c-489f-afd4-0251b6a768a7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.600543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.601522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.601911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.602270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] PATH_INFO: `/v3/projects/e2ecd712d233455ba0ecc4033ebbd88d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.603573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ecd712d233455ba0ecc4033ebbd88d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.605779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.637847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.638921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.639214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.639489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] PATH_INFO: `/v3/users/95c676a7004c403b92ab4f769aa7663e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.640719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c676a7004c403b92ab4f769aa7663e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.642675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.655301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ecd712d233455ba0ecc4033ebbd88d', 'request_id': 'req-90b01833-3e97-49e8-b257-2df45afc9dce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.664174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ecd712d233455ba0ecc4033ebbd88d', 'request_id': 'req-90b01833-3e97-49e8-b257-2df45afc9dce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.670911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ecd712d233455ba0ecc4033ebbd88d', 'request_id': 'req-90b01833-3e97-49e8-b257-2df45afc9dce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.686874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] The token cache is being invalidate because project e2ecd712d233455ba0ecc4033ebbd88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:04.687221 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.688093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.689877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 983/1950] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:04 2022] DELETE /identity/v3/projects/e2ecd712d233455ba0ecc4033ebbd88d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:04.690511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.691909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.705764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c676a7004c403b92ab4f769aa7663e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.722176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c676a7004c403b92ab4f769aa7663e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.733135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c676a7004c403b92ab4f769aa7663e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.743255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c676a7004c403b92ab4f769aa7663e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.743923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c676a7004c403b92ab4f769aa7663e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.752307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 968/1951] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:04 2022] DELETE /identity/v3/users/95c676a7004c403b92ab4f769aa7663e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:04.753266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:04.754657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.770202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b01833-3e97-49e8-b257-2df45afc9dce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.817053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.833959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8941267-8da7-4e7a-801d-c850bcce6811 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:04.840698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.843164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.843609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.844076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] PATH_INFO: `/v3/users/75cfef5ef61245eeb1f867c13b6b7b3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.845896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] RBAC: Authorizing `identity:delete_user(user_id=75cfef5ef61245eeb1f867c13b6b7b3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.848440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.874647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:04.875438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:04.875786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:04.876131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:04.878431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:04.905524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:04.907113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:04.909578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:04.918924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 984/1952] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:04.929617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75cfef5ef61245eeb1f867c13b6b7b3d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.944652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75cfef5ef61245eeb1f867c13b6b7b3d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.958070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75cfef5ef61245eeb1f867c13b6b7b3d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.977062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75cfef5ef61245eeb1f867c13b6b7b3d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.978434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75cfef5ef61245eeb1f867c13b6b7b3d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:04.986480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 969/1953] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:04 2022] DELETE /identity/v3/users/75cfef5ef61245eeb1f867c13b6b7b3d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.239985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.241461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.254245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b836e4-71f5-48b9-bcb0-3c23157beba7 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.372155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.374082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.374442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.374796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] PATH_INFO: `/v3/projects/68a54047a722433f9c8c84bfc4360a74` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.376762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a54047a722433f9c8c84bfc4360a74)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.379624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.384434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.386564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.407795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874a8d-a79b-4aee-8c52-bbc51d67090a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.414976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.416007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.416401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.416785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] PATH_INFO: `/v3/projects/f487c890763249b8afd4cd1f9062010a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.418391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f487c890763249b8afd4cd1f9062010a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.421678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.457856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a54047a722433f9c8c84bfc4360a74', 'request_id': 'req-f3245e6f-894c-45b3-84d5-2d371509c741'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.470979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a54047a722433f9c8c84bfc4360a74', 'request_id': 'req-f3245e6f-894c-45b3-84d5-2d371509c741'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.478569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a54047a722433f9c8c84bfc4360a74', 'request_id': 'req-f3245e6f-894c-45b3-84d5-2d371509c741'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.481320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f487c890763249b8afd4cd1f9062010a', 'request_id': 'req-718d630f-4dd7-45b7-8145-9ff46e627c4a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.490931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f487c890763249b8afd4cd1f9062010a', 'request_id': 'req-718d630f-4dd7-45b7-8145-9ff46e627c4a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.497016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] The token cache is being invalidate because project 68a54047a722433f9c8c84bfc4360a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:05.497310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.498719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.499529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f487c890763249b8afd4cd1f9062010a', 'request_id': 'req-718d630f-4dd7-45b7-8145-9ff46e627c4a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.500964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 985/1954] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:05 2022] DELETE /identity/v3/projects/68a54047a722433f9c8c84bfc4360a74 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.502021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.503504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.528728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] The token cache is being invalidate because project f487c890763249b8afd4cd1f9062010a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:05.529907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.531117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.537399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 970/1955] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:05 2022] DELETE /identity/v3/projects/f487c890763249b8afd4cd1f9062010a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.538688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.546132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.584424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245e6f-894c-45b3-84d5-2d371509c741 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.588561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.619275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d630f-4dd7-45b7-8145-9ff46e627c4a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.624113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.624977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.625268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.625546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] PATH_INFO: `/v3/users/41a9f67bc5aa4480a76f71bb6254b284` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.626832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a9f67bc5aa4480a76f71bb6254b284)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.628758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.645025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.645919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.646206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.646482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] PATH_INFO: `/v3/projects/6997aaac828a4506891b92b842bd339e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.647664 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6997aaac828a4506891b92b842bd339e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.649554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.667178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:05.680683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a9f67bc5aa4480a76f71bb6254b284'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.694635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a9f67bc5aa4480a76f71bb6254b284'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.702736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a9f67bc5aa4480a76f71bb6254b284'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.712401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a9f67bc5aa4480a76f71bb6254b284'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.713586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a9f67bc5aa4480a76f71bb6254b284'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.713771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6997aaac828a4506891b92b842bd339e', 'request_id': 'req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.722638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 971/1956] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:05 2022] DELETE /identity/v3/users/41a9f67bc5aa4480a76f71bb6254b284 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.725598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6997aaac828a4506891b92b842bd339e', 'request_id': 'req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.726444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.728320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.732786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6997aaac828a4506891b92b842bd339e', 'request_id': 'req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.755044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] The token cache is being invalidate because project 6997aaac828a4506891b92b842bd339e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:05.755342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.756023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1b15de5c-f0e1-4864-ab0a-a59c79dc0a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.758032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 986/1957] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:05 2022] DELETE /identity/v3/projects/6997aaac828a4506891b92b842bd339e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.764432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8758250-4524-43e8-8095-0b459323d985 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.764730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8758250-4524-43e8-8095-0b459323d985 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.765032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8758250-4524-43e8-8095-0b459323d985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.785305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d8758250-4524-43e8-8095-0b459323d985 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:05.786604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8758250-4524-43e8-8095-0b459323d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.824616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d00ad-e9f8-4d35-badf-c3b6478ca18f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.834947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.836748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.837316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.837902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] PATH_INFO: `/v3/users/35b091d8b8334fcea70572fb95f8bca5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.840162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b091d8b8334fcea70572fb95f8bca5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.844541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.861712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 987/1958] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:05.872740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.873026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.873422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.893958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:05.895166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.900972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b091d8b8334fcea70572fb95f8bca5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.903877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 988/1959] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:05.908970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.918168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b091d8b8334fcea70572fb95f8bca5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.920652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1bb2b2-6e40-47ab-90f0-1bdcea1d5ad4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:05.926218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:05.926961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b091d8b8334fcea70572fb95f8bca5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.927097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:05.927440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:05.927710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:05.928525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:05.928915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:05.930883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:05.937614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b091d8b8334fcea70572fb95f8bca5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.938610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b091d8b8334fcea70572fb95f8bca5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:05.946989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 972/1960] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:05 2022] DELETE /identity/v3/users/35b091d8b8334fcea70572fb95f8bca5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:05.949367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 989/1961] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:05.950532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:05.953186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:05.998507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:06.038585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a37bc-1ef6-4a3a-95f4-bfd3efff8813 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.043979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.044902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.045396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.045811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.047476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.048734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.082547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.084842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.085741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 973/1962] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:05 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:06.090050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.112471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5221b2-0810-4b9d-b568-9190454f0cba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.136424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:06.178300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfb187-44f6-4b61-aaf3-154dbf2dc548 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.183927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.184615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.187837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.187837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.187837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.188700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.228440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 974/1963] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:06.228635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.228748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.228862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.228862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.228862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] PATH_INFO: `/v3/projects/d3ecc7b433734aea8b9377d4d37f8844` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.228862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ecc7b433734aea8b9377d4d37f8844)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.230001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.236713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058e8cb-529b-417a-8c61-4c9d1a259d8e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.242719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.243414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.243744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.244027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.257023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.258929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.273131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 975/1964] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:06.278044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.297575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2632-0959-4845-b894-11101021c008 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.318232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.318962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.319259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.319389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ecc7b433734aea8b9377d4d37f8844', 'request_id': 'req-aba13f46-1d08-4330-a7cb-b7fc994c95bf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.319766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.320287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.326994 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.337414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ecc7b433734aea8b9377d4d37f8844', 'request_id': 'req-aba13f46-1d08-4330-a7cb-b7fc994c95bf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.348856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ecc7b433734aea8b9377d4d37f8844', 'request_id': 'req-aba13f46-1d08-4330-a7cb-b7fc994c95bf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.355215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 976/1965] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:06.359958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.380503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942c8d4-7ecd-45e8-b219-cae3e06da13f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.403845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] The token cache is being invalidate because project d3ecc7b433734aea8b9377d4d37f8844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:06.406819 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.408863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-aba13f46-1d08-4330-a7cb-b7fc994c95bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.414484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 990/1966] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:06 2022] DELETE /identity/v3/projects/d3ecc7b433734aea8b9377d4d37f8844 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:06.417224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.417715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.418208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.462857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.464234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.507986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524fe34-54b7-4d83-8779-0c2ad857f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.508914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524fe34-54b7-4d83-8779-0c2ad857f122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.509206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524fe34-54b7-4d83-8779-0c2ad857f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.509495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524fe34-54b7-4d83-8779-0c2ad857f122 admin admin] PATH_INFO: `/v3/projects/f6fb5fc3024f428d9d052371dbe90737/users/163fb6502e0442c7aefc0c122ed4c035/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.526963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524fe34-54b7-4d83-8779-0c2ad857f122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fb5fc3024f428d9d052371dbe90737, user_id=163fb6502e0442c7aefc0c122ed4c035, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.530238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524fe34-54b7-4d83-8779-0c2ad857f122 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.540643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 991/1967] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:06.541900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.543194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.554729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 977/1968] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:06 2022] PUT /identity/v3/projects/f6fb5fc3024f428d9d052371dbe90737/users/163fb6502e0442c7aefc0c122ed4c035/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:06.555883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692454fa-b1c8-4bed-95db-7ab163faf3de None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.556183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692454fa-b1c8-4bed-95db-7ab163faf3de None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.556467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692454fa-b1c8-4bed-95db-7ab163faf3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.557383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c461a57-abca-413d-8847-0f04fb8f6ce6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.569574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:06.577155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-692454fa-b1c8-4bed-95db-7ab163faf3de None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.578981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-692454fa-b1c8-4bed-95db-7ab163faf3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.614925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:06.630245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.631381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.631756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.631948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] PATH_INFO: `/v3/projects/3ed1e2ad104545e7bad3960f38b77b85` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.633724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed1e2ad104545e7bad3960f38b77b85)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.635596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.659246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 978/1969] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:06.661605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.662165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.662458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.683141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.685388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.696489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 979/1970] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:06.698401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.713639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7a38b-a15c-4faa-8afb-6ed024cc54fa None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.722505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed1e2ad104545e7bad3960f38b77b85', 'request_id': 'req-1aba54d8-eff1-49e8-bab2-4c4c555152e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.735467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed1e2ad104545e7bad3960f38b77b85', 'request_id': 'req-1aba54d8-eff1-49e8-bab2-4c4c555152e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.745652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed1e2ad104545e7bad3960f38b77b85', 'request_id': 'req-1aba54d8-eff1-49e8-bab2-4c4c555152e3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.761644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] The token cache is being invalidate because project 3ed1e2ad104545e7bad3960f38b77b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:06.761869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.762573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:06.765754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 992/1971] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:06 2022] DELETE /identity/v3/projects/3ed1e2ad104545e7bad3960f38b77b85 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:06.766602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.768827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.787592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.788264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.788551 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.788829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.789560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:06.789903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.791495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf310f-a21d-4a2a-80dc-3ebc513a16a3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.804304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 980/1972] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:06.805422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.805744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.806031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.824896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.826175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.828124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba54d8-eff1-49e8-bab2-4c4c555152e3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.831988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.832643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.832937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.833222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.834439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 981/1973] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:06.835024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.841821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.843239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.849008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.850538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720d8a-1ea6-40d9-998d-d4cbfc7760db service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.854720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82402e-9efb-43ce-baab-41d61ae4d344 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.860009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.860676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.860966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.861244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.861416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 993/1974] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:06.861975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.862727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.863072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.863293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.863862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.883514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:06.884705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:06.894376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 994/1975] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:06.895221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 982/1976] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:06.898484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.899093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:06.909576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a2cfe-0e1f-4b72-8844-64c3d929c525 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.976612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf526f-b867-4995-95d8-b37a120f3652 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:06.978020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.978700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.978985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.979294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.980017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:06.980381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.981944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.982643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:06.983439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:06.983809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:06.984156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:06.984936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:06.986950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:06.996509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 995/1977] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:07.002031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.005009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.014830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7586ba-5aab-4b41-9cc3-0bf1c6c9db56 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.019839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 983/1978] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:06 2022] POST /identity/v3/users => generated 281 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:07.023343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.023645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.024034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.024331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.024669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.025324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.027547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.037893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff4d20-84b0-452d-85d4-370b82f7b509 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.039859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.044181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.045756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.046412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.046734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.047670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.049693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.065534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 984/1979] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:07.070072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 996/1980] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/projects => generated 389 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:07.070244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.071299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.081115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fb3f9-7cf8-42f7-8a6d-92434c00dfc5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.145754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef8b5-7ed2-44e9-aff3-eec738c02364 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.152600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.153304 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.153621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.153925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.155833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.157989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.161315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.162160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.162456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.162799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.163549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.165211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.182351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 985/1981] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:07.188671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.194331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 997/1982] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:07.195376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.200740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1afe04-a319-489f-9ac0-4a77aed408f3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.207517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778dfee-aae9-4c6d-92e6-2f4ff6bfec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.208552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778dfee-aae9-4c6d-92e6-2f4ff6bfec79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.208947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778dfee-aae9-4c6d-92e6-2f4ff6bfec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.209321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778dfee-aae9-4c6d-92e6-2f4ff6bfec79 admin admin] PATH_INFO: `/v3/projects/f8dd9bceeb12418d89553906357a54ba/users/3aae0107d8ef4892b46dfb0b88669c2c/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.209437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ddb38-c01d-4f89-ab32-25ecb8684684 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.216345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.217902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.218270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.218468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.219273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.221019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.227998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778dfee-aae9-4c6d-92e6-2f4ff6bfec79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dd9bceeb12418d89553906357a54ba, user_id=3aae0107d8ef4892b46dfb0b88669c2c, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.231609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778dfee-aae9-4c6d-92e6-2f4ff6bfec79 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.250667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 998/1983] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:07.253833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.264677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 986/1984] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:07 2022] PUT /identity/v3/projects/f8dd9bceeb12418d89553906357a54ba/users/3aae0107d8ef4892b46dfb0b88669c2c/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:07.267914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.268327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.268796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.281196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17749f2-06e3-4fe8-90ef-ea4d2508af5c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.304617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] MFA Rules not processed for user `3aae0107d8ef4892b46dfb0b88669c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:07.305881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.356401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.357097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.357605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.357934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.358683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.360764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.375247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 999/1985] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:07.379487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.391061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9fb5a-3717-4561-81dc-d1206391c0c7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.396317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc923373-ae69-46d5-8d09-26681ef3bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.397237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc923373-ae69-46d5-8d09-26681ef3bf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.397784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc923373-ae69-46d5-8d09-26681ef3bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.397923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 987/1986] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:07.398146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc923373-ae69-46d5-8d09-26681ef3bf8f admin admin] PATH_INFO: `/v3/projects/ac35a47a46574022a64410dee49e900f/users/f2a69269cac149fe98092d9094ff1f06/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.416213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc923373-ae69-46d5-8d09-26681ef3bf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac35a47a46574022a64410dee49e900f, user_id=f2a69269cac149fe98092d9094ff1f06, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.419234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc923373-ae69-46d5-8d09-26681ef3bf8f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.444478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1000/1987] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:07 2022] PUT /identity/v3/projects/ac35a47a46574022a64410dee49e900f/users/f2a69269cac149fe98092d9094ff1f06/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:07.448552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.454785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.455434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.455809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.456164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.494625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] MFA Rules not processed for user `f2a69269cac149fe98092d9094ff1f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:07.496192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:07.528306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.530100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:07.539492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd650-7271-42b7-b1e7-4e6fe1c2b26a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.593814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1001/1988] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:07.606650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.631898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.632879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.633244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.633625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.730231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.733944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.744346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813da470-3931-4920-9c83-a4b478f7b403 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.751481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.752682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.752682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.753839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.754728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.757098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.797273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 988/1989] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:07.799230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.825871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229be225-93ed-42fc-8f5d-97a12e725387 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.842987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.844119 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.844390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.844772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] PATH_INFO: `/v3/users/3aae0107d8ef4892b46dfb0b88669c2c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.845030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1002/1990] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/projects => generated 389 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:07.849195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aae0107d8ef4892b46dfb0b88669c2c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.849435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.851178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.933053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09d807-bf9d-488b-b2e6-6ebca399b05d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:07.937022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aae0107d8ef4892b46dfb0b88669c2c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.940215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:07.940923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:07.941216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:07.941500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:07.942552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:07.944602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:07.964381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aae0107d8ef4892b46dfb0b88669c2c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.974098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1003/1991] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:07 2022] POST /identity/v3/users => generated 301 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:07.976054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aae0107d8ef4892b46dfb0b88669c2c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:07.977172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:07.987520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aae0107d8ef4892b46dfb0b88669c2c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.000990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf07f7b-ac09-445f-bbba-d8b7c0cf9626 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.003372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc46cf74-49c8-4ca7-81d5-f05d44a2d666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aae0107d8ef4892b46dfb0b88669c2c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:08.008541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.009576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.010063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.010495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.011862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.014675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.016445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 989/1992] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:07 2022] DELETE /identity/v3/users/3aae0107d8ef4892b46dfb0b88669c2c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:08.021009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.021399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.021795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.035141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1004/1993] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:08.036331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.038609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.052285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:08.054645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.078711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052911d-e0b7-4475-b7f9-bc3b54d69186 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.117880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:08.159012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:08.204703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 990/1994] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:08.220980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.221979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.223342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.223455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.224136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.225097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] PATH_INFO: `/v3/projects/984623749f2b4c1b9b5d4d6518c4127c/users/6c8c425d5cca431b91c5125aaedad0f2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.245441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=984623749f2b4c1b9b5d4d6518c4127c, user_id=6c8c425d5cca431b91c5125aaedad0f2, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.249343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.266668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1005/1995] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:08 2022] PUT /identity/v3/projects/984623749f2b4c1b9b5d4d6518c4127c/users/6c8c425d5cca431b91c5125aaedad0f2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:08.270335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.311738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6333fb-ec94-43b3-aaee-56cbaf657d32 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.373983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102d24c-fe65-4ca1-9568-96cfdb5656dc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.381762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.383736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.384295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.385118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.386137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.390000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.397214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6193b89-0e25-4b6d-bfb3-ee870260d36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.398412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193b89-0e25-4b6d-bfb3-ee870260d36b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.398528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193b89-0e25-4b6d-bfb3-ee870260d36b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.398742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193b89-0e25-4b6d-bfb3-ee870260d36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.405089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1006/1996] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:08.410215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.412576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6193b89-0e25-4b6d-bfb3-ee870260d36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.414070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6193b89-0e25-4b6d-bfb3-ee870260d36b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.425240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9ed5-4ef3-46c4-90b8-d32ae849b097 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.428960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 991/1997] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:08.451999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.452869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.453160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.453442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] PATH_INFO: `/v3/projects/984623749f2b4c1b9b5d4d6518c4127c/users/6c8c425d5cca431b91c5125aaedad0f2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.458550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=984623749f2b4c1b9b5d4d6518c4127c, user_id=6c8c425d5cca431b91c5125aaedad0f2, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.461611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.486684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1007/1998] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:08 2022] PUT /identity/v3/projects/984623749f2b4c1b9b5d4d6518c4127c/users/6c8c425d5cca431b91c5125aaedad0f2/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:08.487989 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.488362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.488652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.532182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] MFA Rules not processed for user `6c8c425d5cca431b91c5125aaedad0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:08.533556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.634858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 992/1999] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:08.642544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:08.652613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:08.752698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2194a3-9377-4064-93b8-7def56c8db6b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:08.758062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:08.759706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:08.760086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:08.760467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:08.802583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:08.804349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:08.818751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1008/2000] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:09.041415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.044006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.103165 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.191693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.274672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a02659-9e82-4a56-b7b7-b6037fc50fc3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.299319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.301338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.301896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.303140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] PATH_INFO: `/v3/projects/f8dd9bceeb12418d89553906357a54ba` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.304572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dd9bceeb12418d89553906357a54ba)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.309935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.342142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.384351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36607dc7-20ef-463d-bf1b-ab100aab2801 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.393632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.395655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.396145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.396444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] PATH_INFO: `/v3/users/f2a69269cac149fe98092d9094ff1f06` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.397921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a69269cac149fe98092d9094ff1f06)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.400070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.433564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dd9bceeb12418d89553906357a54ba', 'request_id': 'req-6461ebbf-dc01-426f-889d-55576acd3ee5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.450845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dd9bceeb12418d89553906357a54ba', 'request_id': 'req-6461ebbf-dc01-426f-889d-55576acd3ee5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.466805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dd9bceeb12418d89553906357a54ba', 'request_id': 'req-6461ebbf-dc01-426f-889d-55576acd3ee5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.483633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] The token cache is being invalidate because project f8dd9bceeb12418d89553906357a54ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:09.485338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.485338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6461ebbf-dc01-426f-889d-55576acd3ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.491868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 993/2001] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:09 2022] DELETE /identity/v3/projects/f8dd9bceeb12418d89553906357a54ba => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:09.493431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a69269cac149fe98092d9094ff1f06'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.493602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd92fe-bc3e-416d-bed5-a15433c65495 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.494044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd92fe-bc3e-416d-bed5-a15433c65495 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.494429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd92fe-bc3e-416d-bed5-a15433c65495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.496038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a69269cac149fe98092d9094ff1f06'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.516583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a69269cac149fe98092d9094ff1f06'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.516892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a7bd92fe-bc3e-416d-bed5-a15433c65495 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.523093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a7bd92fe-bc3e-416d-bed5-a15433c65495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.532656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a69269cac149fe98092d9094ff1f06'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.547952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3de8f7ba-81db-4cb6-a362-271639a06d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a69269cac149fe98092d9094ff1f06'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:09.584780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1009/2002] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:09 2022] DELETE /identity/v3/users/f2a69269cac149fe98092d9094ff1f06 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:09.587746 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.588421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.588852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.680179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 994/2003] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:09.686075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.688803 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.689230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.689642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.690724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.730183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:09.731641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.808312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:09.850003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1010/2004] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:09.855091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 995/2005] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:09.859989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.870285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c5661-420b-4db4-804a-184cd2966b94 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.884373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.885918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:09.897021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.898055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.898369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.898633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.901786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:09.903427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.907000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:09.926975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1011/2006] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:09.931683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:09.945753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03b621-c1e6-417f-b53e-961e938aa64d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:09.952933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:09.954023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:09.954300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:09.954641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:09.955528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:09.958408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.006290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4df71-f69c-4808-ada7-20d99b270646 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.007893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1012/2007] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:09 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:10.012817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.074783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:10.104188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.104878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.105174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.105461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.128021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437aad-91ec-4d34-a93c-d74f04a5201a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.136319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.138166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.139985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.140511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.141420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.143713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.191656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1013/2008] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:10 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:10.191868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.244164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e5830-3644-475a-8ead-90fb0598de17 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.255084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.255957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.256343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.256880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.257677 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.259503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.274048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1014/2009] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:10.280968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.309307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.312586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.313581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a9489-2280-49cb-ae67-0764d48bb746 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.322780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dfef1-abc9-4ee4-8c3d-e99e28115906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.324395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dfef1-abc9-4ee4-8c3d-e99e28115906 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.324647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dfef1-abc9-4ee4-8c3d-e99e28115906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.325314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dfef1-abc9-4ee4-8c3d-e99e28115906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.331045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dfef1-abc9-4ee4-8c3d-e99e28115906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.335686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dfef1-abc9-4ee4-8c3d-e99e28115906 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.338047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 996/2010] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:10.365872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1015/2011] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:10.369944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:10.386317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aa1fb-0f3d-462e-92a1-ace24695ef38 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:10.398143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:10.399279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.399747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.400221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] PATH_INFO: `/v3/projects/7e92d214c38a458ab35c78997598a2e8/users/c9658d132def4efcba8eecb945089ac9/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.429825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e92d214c38a458ab35c78997598a2e8, user_id=c9658d132def4efcba8eecb945089ac9, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:10.432714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:10.477868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 997/2012] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:10 2022] PUT /identity/v3/projects/7e92d214c38a458ab35c78997598a2e8/users/c9658d132def4efcba8eecb945089ac9/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:10.488666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:10.489224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:10.489784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:10.565381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] MFA Rules not processed for user `c9658d132def4efcba8eecb945089ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:10.567084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:10.741634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1016/2013] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:11.112671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:11.115499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:11.174246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:11.224846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f76e7c-cf23-456d-8859-7722702b7a34 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:11.230064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:11.231345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.231648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.231987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.251914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.253691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:11.300322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 998/2014] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:11.676996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:11.678842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:11.896441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394f74-dd50-4074-9d64-dcc708ec2ff1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:11.910207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:11.911320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:11.913681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:11.914162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] PATH_INFO: `/v3/projects/ac35a47a46574022a64410dee49e900f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:11.919359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac35a47a46574022a64410dee49e900f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:11.928402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.025968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:12.113860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac35a47a46574022a64410dee49e900f', 'request_id': 'req-9bef8566-852b-42be-af5b-1c0e7ad54eff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.149183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac35a47a46574022a64410dee49e900f', 'request_id': 'req-9bef8566-852b-42be-af5b-1c0e7ad54eff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.178316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac35a47a46574022a64410dee49e900f', 'request_id': 'req-9bef8566-852b-42be-af5b-1c0e7ad54eff'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.215091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] The token cache is being invalidate because project ac35a47a46574022a64410dee49e900f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:12.215523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.217728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.223029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1017/2015] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:11 2022] DELETE /identity/v3/projects/ac35a47a46574022a64410dee49e900f => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:12.224077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:12.252560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b13f-a879-4eac-8198-3652482cc191 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:12.478323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:12.480135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:12.480576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:12.481001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] PATH_INFO: `/v3/users/6c8c425d5cca431b91c5125aaedad0f2` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:12.484191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8c425d5cca431b91c5125aaedad0f2)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:12.486933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:12.570309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8c425d5cca431b91c5125aaedad0f2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.608307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8c425d5cca431b91c5125aaedad0f2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.624621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8c425d5cca431b91c5125aaedad0f2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.643741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8c425d5cca431b91c5125aaedad0f2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.644449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1dac291b-63a8-4424-9afb-32a488e3c96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8c425d5cca431b91c5125aaedad0f2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:12.651639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 999/2016] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:12 2022] DELETE /identity/v3/users/6c8c425d5cca431b91c5125aaedad0f2 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:13.382421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:13.384013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:13.411434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8566-852b-42be-af5b-1c0e7ad54eff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:13.676145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:13.677343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:13.677463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:13.677784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] PATH_INFO: `/v3/projects/984623749f2b4c1b9b5d4d6518c4127c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:13.686915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=984623749f2b4c1b9b5d4d6518c4127c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:13.689569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:13.809960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984623749f2b4c1b9b5d4d6518c4127c', 'request_id': 'req-49227c4d-177a-460f-a5ee-8a098d5f1465'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:13.832814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984623749f2b4c1b9b5d4d6518c4127c', 'request_id': 'req-49227c4d-177a-460f-a5ee-8a098d5f1465'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:13.882254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984623749f2b4c1b9b5d4d6518c4127c', 'request_id': 'req-49227c4d-177a-460f-a5ee-8a098d5f1465'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:13.940887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] The token cache is being invalidate because project 984623749f2b4c1b9b5d4d6518c4127c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:13.941508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:13.942581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-49227c4d-177a-460f-a5ee-8a098d5f1465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:13.945751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1018/2017] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:13 2022] DELETE /identity/v3/projects/984623749f2b4c1b9b5d4d6518c4127c => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:13.950742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:13.951518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:13.951518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:13.986482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:13.991571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:14.112909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:14.173276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1000/2018] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:14.181639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.182004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.182281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.208572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:14.209847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:14.221005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1019/2019] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:14.225799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.239842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828acfd4-d101-418d-942c-1f42aaaaa6e8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.247890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.249629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.250243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.250677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.252198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:14.252697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.255247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:14.281537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.289327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1001/2020] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:14.295575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460d414-a0aa-4c0b-ade4-6a7a8193f16f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.306957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.307662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.307957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.308249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.308961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.315413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:14.377637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1020/2021] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:14 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:14.382528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.401025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09127adf-03a2-4a37-9b78-e5f1b94f0754 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.467867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.470720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:14.522233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:14.643911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.647074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.649607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.649950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.652181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.656038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d314d-5439-43af-b76f-60c2614d6e18 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.659271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:14.662023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.663224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.663613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.664019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] PATH_INFO: `/v3/users/78e0bd33c7414863bc6f719c581e5829` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.665483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e0bd33c7414863bc6f719c581e5829)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.668096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:14.717875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1002/2022] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:14 2022] POST /identity/v3/users => generated 291 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:14.719875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.738298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cbe24-e6a4-459e-a9f0-f24d4c159df5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.753397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.754181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.754476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.754761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.755500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.757113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:14.765990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e0bd33c7414863bc6f719c581e5829'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:14.767131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e0bd33c7414863bc6f719c581e5829'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:14.770748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1003/2023] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:14.775134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.777571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:14.779766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e0bd33c7414863bc6f719c581e5829'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:14.780853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e0bd33c7414863bc6f719c581e5829'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:14.792912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09702ed-28f4-4703-874b-7cdb0fdff608 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.794023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a0faa60-e0c4-4021-acce-280942dcca65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e0bd33c7414863bc6f719c581e5829'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:14.799591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.800249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.800480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.800921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.802337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.803317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:14.821840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1004/2024] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:14.827001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:14.838329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1021/2025] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:14 2022] DELETE /identity/v3/users/78e0bd33c7414863bc6f719c581e5829 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:14.840315 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.840522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3361cf2-c751-49b0-beb9-6c6b1539bb95 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:14.841032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.841387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.873601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:14.875828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:14.945412 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca49c8d-b1ef-461d-b878-802e9992791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:14.946506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca49c8d-b1ef-461d-b878-802e9992791c admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:14.947072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca49c8d-b1ef-461d-b878-802e9992791c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:14.953163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca49c8d-b1ef-461d-b878-802e9992791c admin admin] PATH_INFO: `/v3/projects/90ef66cd4e5a4168820fc9f66caa19c8/users/cfae174c5ef34ff68ec6b6bc3ca0d561/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:14.981508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca49c8d-b1ef-461d-b878-802e9992791c admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ef66cd4e5a4168820fc9f66caa19c8, user_id=cfae174c5ef34ff68ec6b6bc3ca0d561, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:14.988097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca49c8d-b1ef-461d-b878-802e9992791c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.034374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1005/2026] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:14 2022] PUT /identity/v3/projects/90ef66cd4e5a4168820fc9f66caa19c8/users/cfae174c5ef34ff68ec6b6bc3ca0d561/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:15.049575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.049997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.050467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.108782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1022/2027] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:15.119140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] MFA Rules not processed for user `cfae174c5ef34ff68ec6b6bc3ca0d561`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:15.119772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.120658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:15.122475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:15.140254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:15.216035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:15.229058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1006/2028] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:15.234594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.266453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabea460-b1a2-447a-9c97-b3fcc179153b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.353614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.354572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.354877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.355229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.385303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed375-8090-4fd2-ab1d-3188caa87c4d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.397830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.398518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.398759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.399110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.399741 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.406648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.435508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.441440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.452121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1007/2029] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:15 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:15.456272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.466323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1023/2030] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:15.549067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e798-21d8-4fe0-b81b-cd42ea1a50b9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.555555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.556500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.556883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.557255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.558277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.562734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.610572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1008/2031] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:15 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:15.614730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.637274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270fc96-1fd4-44d7-b4f0-757fe009ec73 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.652899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.653782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.654911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.655567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.657342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.668423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.698503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1024/2032] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:15.702547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.720212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c82d-cf65-44d4-9829-b276f4c6633b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.755365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.757195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.757626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.757946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] PATH_INFO: `/v3/projects/bde9c278c5384e6d9331b33d3b0a882f/users/c7a7a014e19b429f8602ecd51d27b4d3/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.812724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde9c278c5384e6d9331b33d3b0a882f, user_id=c7a7a014e19b429f8602ecd51d27b4d3, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.820636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:15.862228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1009/2033] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:15 2022] PUT /identity/v3/projects/bde9c278c5384e6d9331b33d3b0a882f/users/c7a7a014e19b429f8602ecd51d27b4d3/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:15.866076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:15.882061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fc27e-84ca-4cf7-abac-f5513b78d679 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:15.985837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c37fb-de1c-461d-857f-c3ef31105118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:15.986703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c37fb-de1c-461d-857f-c3ef31105118 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:15.987088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c37fb-de1c-461d-857f-c3ef31105118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:15.987465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c37fb-de1c-461d-857f-c3ef31105118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:15.988307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c37fb-de1c-461d-857f-c3ef31105118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:15.990039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c37fb-de1c-461d-857f-c3ef31105118 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.021788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1025/2034] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:16.021907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.038030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b74e3-c167-4137-aa48-ba3b3bf9d20e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.049682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f9074-657e-4319-a0da-03d356698b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.050607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f9074-657e-4319-a0da-03d356698b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.050901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f9074-657e-4319-a0da-03d356698b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.051187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f9074-657e-4319-a0da-03d356698b47 admin admin] PATH_INFO: `/v3/projects/bde9c278c5384e6d9331b33d3b0a882f/users/c7a7a014e19b429f8602ecd51d27b4d3/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.058423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f9074-657e-4319-a0da-03d356698b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde9c278c5384e6d9331b33d3b0a882f, user_id=c7a7a014e19b429f8602ecd51d27b4d3, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.061239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f9074-657e-4319-a0da-03d356698b47 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.064638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:16.097577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1010/2035] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:16 2022] PUT /identity/v3/projects/bde9c278c5384e6d9331b33d3b0a882f/users/c7a7a014e19b429f8602ecd51d27b4d3/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:16.099357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.100385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.100385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.163927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] MFA Rules not processed for user `c7a7a014e19b429f8602ecd51d27b4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:16.164233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.164718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.165182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.166995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.201554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:16.225870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] MFA Rules not processed for user `163fb6502e0442c7aefc0c122ed4c035`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:16.227379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.322674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1011/2036] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:16.324628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.328099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.335705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1026/2037] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:16.337128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.339293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.439022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c7879-1f6d-45f4-9cb7-2628d314797d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.453291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.454038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.454324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.454683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.457878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.459859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.462558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0e627-86d5-46f9-bb49-5808c17e5ddd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.468715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.469607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.469922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.470198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] PATH_INFO: `/v3/projects/32bed34e68944ec3b11676de475afe75` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.472507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bed34e68944ec3b11676de475afe75)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.476472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.501522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1027/2038] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:16 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:16.505938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.508201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.554678 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bed34e68944ec3b11676de475afe75', 'request_id': 'req-873551c6-61f1-412c-8f76-7b483e83fd5b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.563403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bed34e68944ec3b11676de475afe75', 'request_id': 'req-873551c6-61f1-412c-8f76-7b483e83fd5b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.573034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bed34e68944ec3b11676de475afe75', 'request_id': 'req-873551c6-61f1-412c-8f76-7b483e83fd5b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.597999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] The token cache is being invalidate because project 32bed34e68944ec3b11676de475afe75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:16.599173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.601085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.602970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3f61-cb46-4840-9017-7c93888223b0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.606537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1012/2039] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:16 2022] DELETE /identity/v3/projects/32bed34e68944ec3b11676de475afe75 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:16.608124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.609813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.632450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:16.698924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.699782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.700071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.700349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.704565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.717372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:16.719312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873551c6-61f1-412c-8f76-7b483e83fd5b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.730047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.730677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.730965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.731250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.733335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.734140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.765802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1013/2040] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:16 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:16.765802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.768487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.783326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023327b0-ccff-43dc-909e-11555272e886 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.788519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.789419 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.789729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.790023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] PATH_INFO: `/v3/users/eba74be9499c4595ae5b2b9bd451b4bd` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.791697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba74be9499c4595ae5b2b9bd451b4bd)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.793735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.833526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.836111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.851616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1028/2041] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:16.852725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.901046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94a214-ae71-449e-b7ef-7eb44d2f4c13 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:16.913689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba74be9499c4595ae5b2b9bd451b4bd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.918252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:16.921080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:16.922402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:16.922688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:16.923450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:16.929940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:16.954976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba74be9499c4595ae5b2b9bd451b4bd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.963418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba74be9499c4595ae5b2b9bd451b4bd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.972164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba74be9499c4595ae5b2b9bd451b4bd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.973081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba74be9499c4595ae5b2b9bd451b4bd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:16.973948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1029/2042] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:16.977585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:16.980728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:16.981144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1014/2043] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:16 2022] DELETE /identity/v3/users/eba74be9499c4595ae5b2b9bd451b4bd => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:17.010348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53e2f6-b466-4f0c-9708-dee1b021c715 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:17.140020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:17.143309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:17.145178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:17.145635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] PATH_INFO: `/v3/projects/c6754cfb3b88408eb39872bbbf457d19/users/b25f8f68987848d7864fd0b0119cf86a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:17.189869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6754cfb3b88408eb39872bbbf457d19, user_id=b25f8f68987848d7864fd0b0119cf86a, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:17.202333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:17.236723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1030/2044] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:16 2022] PUT /identity/v3/projects/c6754cfb3b88408eb39872bbbf457d19/users/b25f8f68987848d7864fd0b0119cf86a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:17.240871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:17.262927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce5ad9-4b11-4cf1-b780-89c78f09dcfb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:17.430544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:17.431376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:17.431807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:17.432207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:17.434009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:17.442212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:17.480934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1015/2045] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:17.482362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:17.495862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb511c1-1d33-4117-bec8-e48284ca0fbb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:17.512485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:17.514353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:17.517183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:17.517633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] PATH_INFO: `/v3/projects/c6754cfb3b88408eb39872bbbf457d19/users/b25f8f68987848d7864fd0b0119cf86a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:17.522967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6754cfb3b88408eb39872bbbf457d19, user_id=b25f8f68987848d7864fd0b0119cf86a, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:17.535022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:17.569807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:17.572022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:17.616409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25e0ac-4345-4af5-85c0-1d8c9dbec266 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:17.622182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1031/2046] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:17 2022] PUT /identity/v3/projects/c6754cfb3b88408eb39872bbbf457d19/users/b25f8f68987848d7864fd0b0119cf86a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:17.635272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:17.721981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:17.723370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:17.723973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:17.724734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] PATH_INFO: `/v3/projects/98b17fddfe044d889a3d5e771f25da51` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:17.726404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b17fddfe044d889a3d5e771f25da51)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:17.729196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:17.823243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b17fddfe044d889a3d5e771f25da51', 'request_id': 'req-62920a7a-bc82-42e4-9d04-9f1825825899'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:17.844272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b17fddfe044d889a3d5e771f25da51', 'request_id': 'req-62920a7a-bc82-42e4-9d04-9f1825825899'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:17.860619 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b17fddfe044d889a3d5e771f25da51', 'request_id': 'req-62920a7a-bc82-42e4-9d04-9f1825825899'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:17.887423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:17.889738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] The token cache is being invalidate because project 98b17fddfe044d889a3d5e771f25da51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:17.891032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:17.891899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:17.893515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-62920a7a-bc82-42e4-9d04-9f1825825899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:17.898324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1016/2047] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:17 2022] DELETE /identity/v3/projects/98b17fddfe044d889a3d5e771f25da51 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:17.903176 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184601fa-4e7f-45dd-8044-1b9a82fcf355 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:17.907317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184601fa-4e7f-45dd-8044-1b9a82fcf355 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:17.907542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184601fa-4e7f-45dd-8044-1b9a82fcf355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:17.912985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f54f8c-7b5c-41e2-9c8d-17ef4ef32265 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:17.992004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-184601fa-4e7f-45dd-8044-1b9a82fcf355 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:17.993320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-184601fa-4e7f-45dd-8044-1b9a82fcf355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:18.088458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.093047 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.093396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.096767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] PATH_INFO: `/v3/users/cfae174c5ef34ff68ec6b6bc3ca0d561` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.101115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfae174c5ef34ff68ec6b6bc3ca0d561)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.105352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.109610 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:18.178135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1017/2048] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:18.188535 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.199596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.199596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.249265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:18.251585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:18.257033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfae174c5ef34ff68ec6b6bc3ca0d561'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:18.261269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfae174c5ef34ff68ec6b6bc3ca0d561'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:18.265504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1018/2049] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:18.269468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.286321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfae174c5ef34ff68ec6b6bc3ca0d561'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:18.287035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfae174c5ef34ff68ec6b6bc3ca0d561'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:18.288728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62829d39-9fd1-4381-a594-e6e21cd4991d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.294917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.296533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.297201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.297952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.299344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:18.299896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.302101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.307812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9fad914c-6a50-422a-bc76-6a6a6bf080a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfae174c5ef34ff68ec6b6bc3ca0d561'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:18.321106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1019/2050] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:18.324599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.340698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444149ea-882a-4aaf-b730-862a2589403a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.349196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.350454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.351156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.351677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.352849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.355275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.358763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1032/2051] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:17 2022] DELETE /identity/v3/users/cfae174c5ef34ff68ec6b6bc3ca0d561 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:18.364504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.364899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.365197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.393233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:18.408240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1020/2052] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:18 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:18.411919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.420138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:18.421723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:18.538844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1033/2053] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:18.549044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e11ef4-e08f-4091-b073-7030ef225b9b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.556232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.557032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.557325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.557752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.558372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.562190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.603138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1021/2054] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:18 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:18.611441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.620349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.621078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:18.634722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a49974-d9da-487d-b3a8-79189ebafa2b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.647524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.648671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.651544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.651915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.652731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.657180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.677752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1034/2055] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:18.683261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.721923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a224c-cd2a-4126-b138-d2c9c940583f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.734591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.735793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.736088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.736429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.738416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.759004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0963a-8a89-44ff-8779-f20af84cb423 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.761313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.764467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5baf9-c342-45e4-ad65-210a42fc1c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.766333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5baf9-c342-45e4-ad65-210a42fc1c35 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.766854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5baf9-c342-45e4-ad65-210a42fc1c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.767400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5baf9-c342-45e4-ad65-210a42fc1c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.781476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1035/2056] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:18.787615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5baf9-c342-45e4-ad65-210a42fc1c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.788385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:18.789254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5baf9-c342-45e4-ad65-210a42fc1c35 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.802183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1022/2057] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:18.812512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab0de-d331-4b60-9517-ae387ef8a7e9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:18.826911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:18.827830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.828119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.828401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] PATH_INFO: `/v3/projects/26877e7465ce4d36894696968895a04e/users/5e519d5b8128425099581d6dfe23af33/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:18.862861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=26877e7465ce4d36894696968895a04e, user_id=5e519d5b8128425099581d6dfe23af33, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:18.869197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:18.900174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1036/2058] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:18 2022] PUT /identity/v3/projects/26877e7465ce4d36894696968895a04e/users/5e519d5b8128425099581d6dfe23af33/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:18.914929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:18.916025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:18.917271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:19.020305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] MFA Rules not processed for user `5e519d5b8128425099581d6dfe23af33`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:19.022463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:19.239552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1023/2059] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:19.243705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:19.333713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:19.420215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc684a0-4e60-4e8a-b8ff-e8b48c06f21c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:19.426032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:19.426791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:19.427145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:19.427489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:19.428346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:19.435923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:19.470493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1037/2060] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:19 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:19.470802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:19.490323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2814-67a4-471e-a714-a12483e3cade None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:19.515356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:19.615404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:19.616154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:19.616698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:19.616990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:19.617789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:19.619836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:19.663078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1024/2061] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:19 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:19.668427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:19.681788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531205-6472-401a-90a6-e8206fb934e9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:19.694222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:19.695226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:19.700089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:19.700679 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:19.702385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:19.707294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:19.744314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1038/2062] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:19.744584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:19.762299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136018-fb5e-469f-881d-a04e43f2b285 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:19.774796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:19.776217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:19.776217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:19.776504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] PATH_INFO: `/v3/projects/bd458236d697462c85f5d12c737addb0/users/5bc93eb077dc4f6d9226b291d5c9100f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:19.819897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd458236d697462c85f5d12c737addb0, user_id=5bc93eb077dc4f6d9226b291d5c9100f, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:19.830274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:19.866432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1025/2063] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:19 2022] PUT /identity/v3/projects/bd458236d697462c85f5d12c737addb0/users/5bc93eb077dc4f6d9226b291d5c9100f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:19.871075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:19.890818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3c2ea-1605-41fd-9fd8-f5c126f0e26b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:20.002742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:20.004473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:20.018804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:20.026683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:20.026990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:20.027276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:20.031692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11233ed8-b714-4de5-8a55-1cc323933bae admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:20.032499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:20.038466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:20.049878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:20.051600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:20.051600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:20.051600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] PATH_INFO: `/v3/projects/90ef66cd4e5a4168820fc9f66caa19c8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:20.052902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ef66cd4e5a4168820fc9f66caa19c8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:20.055012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:20.068902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1039/2064] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:20.070235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:20.095487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bf73-67bd-4d39-a010-eb9b01912bd0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:20.109895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ba6f1-6d2a-4b93-a744-c8ad6b1e3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:20.111153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ba6f1-6d2a-4b93-a744-c8ad6b1e3f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:20.111441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ba6f1-6d2a-4b93-a744-c8ad6b1e3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:20.111727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ba6f1-6d2a-4b93-a744-c8ad6b1e3f89 admin admin] PATH_INFO: `/v3/projects/bd458236d697462c85f5d12c737addb0/users/5bc93eb077dc4f6d9226b291d5c9100f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:20.114162 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ba6f1-6d2a-4b93-a744-c8ad6b1e3f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd458236d697462c85f5d12c737addb0, user_id=5bc93eb077dc4f6d9226b291d5c9100f, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:20.120967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ba6f1-6d2a-4b93-a744-c8ad6b1e3f89 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:20.169557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1040/2065] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:20 2022] PUT /identity/v3/projects/bd458236d697462c85f5d12c737addb0/users/5bc93eb077dc4f6d9226b291d5c9100f/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:20.174787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:20.175790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:20.176084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:20.220675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ef66cd4e5a4168820fc9f66caa19c8', 'request_id': 'req-c29b8d68-c05f-4140-b6f1-b1d34f316874'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.236107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ef66cd4e5a4168820fc9f66caa19c8', 'request_id': 'req-c29b8d68-c05f-4140-b6f1-b1d34f316874'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.272192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ef66cd4e5a4168820fc9f66caa19c8', 'request_id': 'req-c29b8d68-c05f-4140-b6f1-b1d34f316874'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.298672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] The token cache is being invalidate because project 90ef66cd4e5a4168820fc9f66caa19c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:20.299732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.300853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.303396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] MFA Rules not processed for user `5bc93eb077dc4f6d9226b291d5c9100f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:20.303517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1026/2066] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:20 2022] DELETE /identity/v3/projects/90ef66cd4e5a4168820fc9f66caa19c8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:20.308228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:20.311518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:20.312131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:20.520150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1041/2067] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:20.534108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:20.535039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b8d68-c05f-4140-b6f1-b1d34f316874 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:20.537361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:20.559308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:20.564502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:20.565078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:20.565377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] PATH_INFO: `/v3/users/c7a7a014e19b429f8602ecd51d27b4d3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:20.568524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a7a014e19b429f8602ecd51d27b4d3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:20.582898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:20.587042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:20.614364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:20.713857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a7a014e19b429f8602ecd51d27b4d3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.727977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9c6ac-063f-4bf7-a205-433d4f7134fa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:20.734052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:20.736246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:20.736586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:20.736906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:20.738459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a7a014e19b429f8602ecd51d27b4d3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.749318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a7a014e19b429f8602ecd51d27b4d3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.757903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:20.759771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:20.767430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a7a014e19b429f8602ecd51d27b4d3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.768381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a7a014e19b429f8602ecd51d27b4d3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:20.780534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1042/2068] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:20.781379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1027/2069] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:20 2022] DELETE /identity/v3/users/c7a7a014e19b429f8602ecd51d27b4d3 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:21.688074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:21.713704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08a07f-2ad8-40fa-8b2a-4d9e8de8f678 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:21.862876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bc93eb077dc4f6d9226b291d5c9100f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd458236d697462c85f5d12c737addb0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:21.864046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:21.864640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:21.865123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:21.866097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:21.867525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:21.869678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:21.889491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1043/2070] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:21.893183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:21.894750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:21.910611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb2dad-faba-468f-a8e0-4acda9b0dcd6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:21.917063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bc93eb077dc4f6d9226b291d5c9100f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd458236d697462c85f5d12c737addb0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:21.917680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:21.918013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:21.918301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:21.919016 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:21.920970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:21.975919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:21.976452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1028/2071] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:21 2022] POST /identity/v3/projects => generated 352 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:21.988475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac84d0e-d287-4bc0-80c9-ae9ae5eccd62 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:22.172025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bc93eb077dc4f6d9226b291d5c9100f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd458236d697462c85f5d12c737addb0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:22.172993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.173386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.173872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.174739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:22.175171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:22.176943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:22.200296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1044/2072] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:22.239159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:22.241109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:22.255654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:22.272923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728d287-b16a-4968-9cd4-0be4d3007169 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:22.281418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bc93eb077dc4f6d9226b291d5c9100f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd458236d697462c85f5d12c737addb0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:22.282604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.283115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.283559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] PATH_INFO: `/v3/projects/b4dd3e9e2ad84cce9b0740c76f84a8c5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.286725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b4dd3e9e2ad84cce9b0740c76f84a8c5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:22.289469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:22.403439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:22.414689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac218bdd-1a90-480c-9c38-9de085228f82 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:22.424029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4dd3e9e2ad84cce9b0740c76f84a8c5', 'request_id': 'req-de91d772-22e7-4cdb-9762-04a048835f45'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.442874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4dd3e9e2ad84cce9b0740c76f84a8c5', 'request_id': 'req-de91d772-22e7-4cdb-9762-04a048835f45'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.456984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4dd3e9e2ad84cce9b0740c76f84a8c5', 'request_id': 'req-de91d772-22e7-4cdb-9762-04a048835f45'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.490300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] The token cache is being invalidate because project b4dd3e9e2ad84cce9b0740c76f84a8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:22.490723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.491925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de91d772-22e7-4cdb-9762-04a048835f45 tempest-VolumeQuotaClassesTest-1047992763 tempest-VolumeQuotaClassesTest-1047992763-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.497412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1045/2073] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:22 2022] DELETE /identity/v3/projects/b4dd3e9e2ad84cce9b0740c76f84a8c5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:22.538772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:22.540160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.540612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.541208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] PATH_INFO: `/v3/projects/bde9c278c5384e6d9331b33d3b0a882f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.542840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde9c278c5384e6d9331b33d3b0a882f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:22.546416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:22.610586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:22.677980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde9c278c5384e6d9331b33d3b0a882f', 'request_id': 'req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.685047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde9c278c5384e6d9331b33d3b0a882f', 'request_id': 'req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.695859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde9c278c5384e6d9331b33d3b0a882f', 'request_id': 'req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.718029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] The token cache is being invalidate because project bde9c278c5384e6d9331b33d3b0a882f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:22.718345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.719085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.721015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1029/2074] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:22 2022] DELETE /identity/v3/projects/bde9c278c5384e6d9331b33d3b0a882f => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:22.726339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e562e89-f8c1-40c4-bead-cda445bf126e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.726875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e562e89-f8c1-40c4-bead-cda445bf126e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.727071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e562e89-f8c1-40c4-bead-cda445bf126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.769329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1e562e89-f8c1-40c4-bead-cda445bf126e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:22.771362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1e562e89-f8c1-40c4-bead-cda445bf126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:22.822394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:22.824104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:22.858102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1046/2075] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:22.866477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.866866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.867236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.891228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:22.892506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:22.905779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1047/2076] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:22.907246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:22.909499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca9130-78d5-46ac-955a-1e47a7eadc7e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:22.915553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:22.916539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.916896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.917245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] PATH_INFO: `/v3/users/5e519d5b8128425099581d6dfe23af33` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.918654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e519d5b8128425099581d6dfe23af33)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:22.921104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:22.922135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bec874-5df4-4899-b95a-d740921b8f1c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:22.928502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:22.929801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.930326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.930867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.932445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:22.933076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:22.935084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:22.950601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1048/2077] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:22.958175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:22.972216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7949c-af6f-4f12-aa5c-b8d58d2fc322 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:22.978281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:22.978949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:22.979236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:22.979515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:22.980217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:22.982191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:22.983019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e519d5b8128425099581d6dfe23af33'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:22.996231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e519d5b8128425099581d6dfe23af33'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:23.007098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e519d5b8128425099581d6dfe23af33'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:23.015437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1049/2078] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:22 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:23.017808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e519d5b8128425099581d6dfe23af33'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:23.018707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-106356de-3daf-46af-aa42-b41943dae980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e519d5b8128425099581d6dfe23af33'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:23.019473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.021222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:23.028553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1030/2079] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:22 2022] DELETE /identity/v3/users/5e519d5b8128425099581d6dfe23af33 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:23.033490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03690d95-5720-4721-ab74-6406688b7698 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.033934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03690d95-5720-4721-ab74-6406688b7698 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.034347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03690d95-5720-4721-ab74-6406688b7698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.052663 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-03690d95-5720-4721-ab74-6406688b7698 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:23.054463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-03690d95-5720-4721-ab74-6406688b7698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:23.077542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:23.117802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfd025-7c3e-41b4-b349-d8336c02b92b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.132731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1031/2080] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:23.143959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03690d95-5720-4721-ab74-6406688b7698 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.145385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-03690d95-5720-4721-ab74-6406688b7698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:23.185990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:23.240006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:23.240823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.240976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.241260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.241995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:23.244159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:23.262964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03690d95-5720-4721-ab74-6406688b7698 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.269523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:23.270906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.271198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.271480 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.293618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1050/2081] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:23 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:23.294369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:23.296914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.298912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:23.320629 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6357d3b-5e10-4550-97c1-db703b4a0121 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.333763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1032/2082] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:23.338717 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:23.339424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.339715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.339999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.340750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:23.343328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:23.369757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1051/2083] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:23.372121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.386274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b697bfc-05d0-4047-9c35-05792b96f329 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.393236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159979b-6811-4d68-bd09-8d13f99bb1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:23.393884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159979b-6811-4d68-bd09-8d13f99bb1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.394172 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159979b-6811-4d68-bd09-8d13f99bb1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.394453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159979b-6811-4d68-bd09-8d13f99bb1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.395195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159979b-6811-4d68-bd09-8d13f99bb1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:23.396854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159979b-6811-4d68-bd09-8d13f99bb1a2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:23.422376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1033/2084] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:23.423921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.442630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ef3eb-6fe8-411d-894c-e978439b7fd5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.461330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:23.462785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.463082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.463363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] PATH_INFO: `/v3/projects/53cde2850758461a80e892bc7cf85187/users/1b9e1c0a19a440ce96133a06b4e76426/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.505817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53cde2850758461a80e892bc7cf85187, user_id=1b9e1c0a19a440ce96133a06b4e76426, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:23.512509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:23.546745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1052/2085] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:23 2022] PUT /identity/v3/projects/53cde2850758461a80e892bc7cf85187/users/1b9e1c0a19a440ce96133a06b4e76426/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:23.558596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.558909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.559194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.595432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] MFA Rules not processed for user `1b9e1c0a19a440ce96133a06b4e76426`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:23.596708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:23.694421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.696462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1034/2086] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:23.784372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6b6ca-ed70-4e1d-9d22-88160b2ecb51 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.791567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:23.792625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:23.793061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:23.793715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:23.795309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:23.797973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:23.853396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1053/2087] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:23 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:23.859577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:23.875283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7351d-f994-476e-95d6-ab7d45b17552 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:23.999192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:24.071009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.075313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.075764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.077354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.079476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.096764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.174427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1035/2088] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:23 2022] POST /identity/v3/users => generated 297 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:24.179127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.194124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4cebd-a891-4427-a83b-c7c97ada79d7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.211875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.212588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.212876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.213410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.213944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.216171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.233860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1054/2089] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:24.236660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.251235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074c4a3-8051-4020-80a6-08a751cef933 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.289761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.290719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.291198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.291312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] PATH_INFO: `/v3/projects/b4c402557b7946b69c364ac64e6a2e95/users/8896fde0a653429f9abaf727ecf187ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.346665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c402557b7946b69c364ac64e6a2e95, user_id=8896fde0a653429f9abaf727ecf187ea, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.351083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.366578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.374140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:24.385151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1036/2090] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:24 2022] PUT /identity/v3/projects/b4c402557b7946b69c364ac64e6a2e95/users/8896fde0a653429f9abaf727ecf187ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:24.385151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.436882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321c78e-5b62-41cc-820d-098a708023f1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.512901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21484bd-09d3-4649-b158-5c439a663dc7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.541620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.542520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.542816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.543206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.543980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.548063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.584688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1037/2091] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:24.586984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.605522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aeb0b8-08b1-4fe5-ac01-1aff388b6bb8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.606815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.608452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.608747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.609010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] PATH_INFO: `/v3/projects/26877e7465ce4d36894696968895a04e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.611567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26877e7465ce4d36894696968895a04e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.616429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.617341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2dd5f-98ce-48d6-a821-e36d224ba906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.618287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2dd5f-98ce-48d6-a821-e36d224ba906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.618574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2dd5f-98ce-48d6-a821-e36d224ba906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.618855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2dd5f-98ce-48d6-a821-e36d224ba906 admin admin] PATH_INFO: `/v3/projects/b4c402557b7946b69c364ac64e6a2e95/users/8896fde0a653429f9abaf727ecf187ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.625079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2dd5f-98ce-48d6-a821-e36d224ba906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c402557b7946b69c364ac64e6a2e95, user_id=8896fde0a653429f9abaf727ecf187ea, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.627769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2dd5f-98ce-48d6-a821-e36d224ba906 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:24.635480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:24.672190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1038/2092] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:24 2022] PUT /identity/v3/projects/b4c402557b7946b69c364ac64e6a2e95/users/8896fde0a653429f9abaf727ecf187ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:24.683512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.683857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.684594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.755116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26877e7465ce4d36894696968895a04e', 'request_id': 'req-8385dc2a-7b6f-4c2c-9121-88ef42d30793'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:24.758879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] MFA Rules not processed for user `8896fde0a653429f9abaf727ecf187ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:24.761301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:24.776125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26877e7465ce4d36894696968895a04e', 'request_id': 'req-8385dc2a-7b6f-4c2c-9121-88ef42d30793'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:24.793952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26877e7465ce4d36894696968895a04e', 'request_id': 'req-8385dc2a-7b6f-4c2c-9121-88ef42d30793'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:24.821642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] The token cache is being invalidate because project 26877e7465ce4d36894696968895a04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:24.822044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:24.827576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:24.829231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1055/2093] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:24 2022] DELETE /identity/v3/projects/26877e7465ce4d36894696968895a04e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:24.833123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.835898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:24.930812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1039/2094] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:24.942060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:24.943529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:24.969647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385dc2a-7b6f-4c2c-9121-88ef42d30793 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:24.981828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:24.983994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:24.984628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:24.985258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] PATH_INFO: `/v3/users/5bc93eb077dc4f6d9226b291d5c9100f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:24.986822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:24.997466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc93eb077dc4f6d9226b291d5c9100f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:24.999925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:25.042868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd387f1-4dad-413d-99d1-10b77168e482 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:25.051670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:25.054984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:25.055785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:25.057246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:25.069411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc93eb077dc4f6d9226b291d5c9100f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.070961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc93eb077dc4f6d9226b291d5c9100f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.076934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:25.079475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:25.082200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc93eb077dc4f6d9226b291d5c9100f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.083121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc93eb077dc4f6d9226b291d5c9100f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.107834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1040/2095] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:25.109597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc93eb077dc4f6d9226b291d5c9100f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.141697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1056/2096] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:24 2022] DELETE /identity/v3/users/5bc93eb077dc4f6d9226b291d5c9100f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:25.229206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:25.245796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5685-e68a-443f-a335-3a071d769908 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:25.378321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:25.379270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:25.379825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:25.380219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:25.382869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:25.486770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:25.488244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:25.515696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1041/2097] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:25.610019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:25.612071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:25.630353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:25.636609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:25.638193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:25.793067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea731c3c-95bf-4bd9-b99d-d63e19ba6bd5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:25.804449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:25.806699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:25.807235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:25.808263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] PATH_INFO: `/v3/users/c9658d132def4efcba8eecb945089ac9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:25.816562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9658d132def4efcba8eecb945089ac9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:25.818513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78100d5-ac16-49a9-a02a-5965905c6927 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:25.820087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:25.824998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:25.827004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:25.827506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:25.827942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] PATH_INFO: `/v3/projects/bd458236d697462c85f5d12c737addb0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:25.830084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd458236d697462c85f5d12c737addb0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:25.832011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:25.876093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:25.908061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9658d132def4efcba8eecb945089ac9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.908794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9658d132def4efcba8eecb945089ac9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.918003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9658d132def4efcba8eecb945089ac9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.918967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9658d132def4efcba8eecb945089ac9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.929206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd458236d697462c85f5d12c737addb0', 'request_id': 'req-de21a126-f3a3-4af9-935d-729154e6ea84'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.939773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5e49c6a-8f29-4116-b1ab-d67e996a9d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9658d132def4efcba8eecb945089ac9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.941163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd458236d697462c85f5d12c737addb0', 'request_id': 'req-de21a126-f3a3-4af9-935d-729154e6ea84'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.962347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd458236d697462c85f5d12c737addb0', 'request_id': 'req-de21a126-f3a3-4af9-935d-729154e6ea84'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.981543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] The token cache is being invalidate because project bd458236d697462c85f5d12c737addb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:25.982487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.983847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-de21a126-f3a3-4af9-935d-729154e6ea84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:25.990444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1042/2098] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:25 2022] DELETE /identity/v3/projects/bd458236d697462c85f5d12c737addb0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:25.992377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6cf2e-a12d-4a58-8ac4-fa41b00d6c21 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:25.992957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6cf2e-a12d-4a58-8ac4-fa41b00d6c21 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:25.993378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6cf2e-a12d-4a58-8ac4-fa41b00d6c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:25.999826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1057/2099] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:25 2022] DELETE /identity/v3/users/c9658d132def4efcba8eecb945089ac9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:26.004486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.004777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.005063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.029943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b0e6cf2e-a12d-4a58-8ac4-fa41b00d6c21 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:26.031729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b0e6cf2e-a12d-4a58-8ac4-fa41b00d6c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.043444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:26.045335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.203224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1058/2100] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:26.204297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.205065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1043/2101] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:26.211770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.212078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.212362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.214523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.251589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79c7b1-b360-41d4-8895-2f0e4e25ee87 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.257242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:26.258511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.261708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.262762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.263602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.263602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] PATH_INFO: `/v3/users/1b9e1c0a19a440ce96133a06b4e76426` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.265493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9e1c0a19a440ce96133a06b4e76426)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.267838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.291426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1044/2102] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:26.292546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.293966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.343457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:26.385907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9e1c0a19a440ce96133a06b4e76426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:26.386645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9e1c0a19a440ce96133a06b4e76426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:26.405763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be052-d05a-4e17-ad0b-4926e7bcabe5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.406007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9e1c0a19a440ce96133a06b4e76426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:26.406705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9e1c0a19a440ce96133a06b4e76426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:26.414383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.415492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.415786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.416072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.418268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.422544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9e1c0a19a440ce96133a06b4e76426'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:26.447352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.455095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1059/2103] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:26 2022] DELETE /identity/v3/users/1b9e1c0a19a440ce96133a06b4e76426 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:26.456140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.457186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d846705-fe6a-4b4a-a0b4-ba8b77d4da74 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.461812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.474272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1045/2104] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:26.490231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.490543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.490839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.541912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:26.543248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.553481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f1396-d2a9-4084-85e3-974ac84db97c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.562202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.563340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.563633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.563949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.564857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:26.565265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.567449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.576631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1046/2105] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:26.586101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1060/2106] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:26.588730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.608075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90d22-2f7d-4e10-b152-345b0fdea9f5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.628156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.628853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.629145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.629429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.630226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.634184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.698742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.701296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:26.713305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1047/2107] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:26 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:26.716132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.801983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:26.879414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045c786-4fcd-40fe-8569-95c5f439f6dc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.899910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77317b09-29cf-4a30-830a-8d356b9063df admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.907332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.908618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.908988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.909354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.910764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.912952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.946900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.948067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1048/2108] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:26 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:26.951474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.952044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.952338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.953047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:26.967632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.971103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82639b8-91eb-491a-83da-190f786d199d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:26.971785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:26.978852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:26.985264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:26.986029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:26.986471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:26.986825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:26.987470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:26.989114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.001015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1049/2109] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:27.005157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.006033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1061/2110] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:27.012311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.016290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.017388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a2dc-c599-4fb3-bd97-94d0d5b8efcf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.022654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.023374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.024218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.024218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.024897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.026883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.036242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ca3f4-376d-4cc1-8ff7-ef2f4ba12722 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.041322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.042258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.042546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.042825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] PATH_INFO: `/v3/projects/7e92d214c38a458ab35c78997598a2e8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.044073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e92d214c38a458ab35c78997598a2e8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.047240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1050/2111] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:27.047852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.051491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.072410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab9312-3364-49ee-ba59-64465a16113d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.080196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871827e-3e55-4d5e-8a14-987b70252820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.081402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871827e-3e55-4d5e-8a14-987b70252820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.082045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871827e-3e55-4d5e-8a14-987b70252820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.082428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871827e-3e55-4d5e-8a14-987b70252820 admin admin] PATH_INFO: `/v3/projects/0e8c0eb611504140a00f13e7cedaf70a/users/30902ec0e5594cca8dff8a7113d5e39a/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.103660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871827e-3e55-4d5e-8a14-987b70252820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8c0eb611504140a00f13e7cedaf70a, user_id=30902ec0e5594cca8dff8a7113d5e39a, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.106716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871827e-3e55-4d5e-8a14-987b70252820 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.157045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1051/2112] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:27 2022] PUT /identity/v3/projects/0e8c0eb611504140a00f13e7cedaf70a/users/30902ec0e5594cca8dff8a7113d5e39a/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:27.169333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.172398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.173070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.211366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e92d214c38a458ab35c78997598a2e8', 'request_id': 'req-a3f33306-b376-46f3-b9ee-14f44130642b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:27.223194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e92d214c38a458ab35c78997598a2e8', 'request_id': 'req-a3f33306-b376-46f3-b9ee-14f44130642b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:27.224742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] MFA Rules not processed for user `30902ec0e5594cca8dff8a7113d5e39a`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:27.226858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.249508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e92d214c38a458ab35c78997598a2e8', 'request_id': 'req-a3f33306-b376-46f3-b9ee-14f44130642b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:27.276288 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] The token cache is being invalidate because project 7e92d214c38a458ab35c78997598a2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:27.277031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:27.278492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a3f33306-b376-46f3-b9ee-14f44130642b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:27.287334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1062/2113] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:27 2022] DELETE /identity/v3/projects/7e92d214c38a458ab35c78997598a2e8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:27.291450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd8fd5-25a9-46ab-b578-4be88be7ebb1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.291450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd8fd5-25a9-46ab-b578-4be88be7ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.291450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd8fd5-25a9-46ab-b578-4be88be7ebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.338526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-47bd8fd5-25a9-46ab-b578-4be88be7ebb1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:27.339893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-47bd8fd5-25a9-46ab-b578-4be88be7ebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.400947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1052/2114] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:27.403037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.408795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.552923 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1063/2115] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:27.564903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189485c-328c-49c6-9020-ba775b72572b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.565389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189485c-328c-49c6-9020-ba775b72572b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.566071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189485c-328c-49c6-9020-ba775b72572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.611503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:27.613013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4189485c-328c-49c6-9020-ba775b72572b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:27.615090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4189485c-328c-49c6-9020-ba775b72572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.631179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1064/2116] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:27.636430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189485c-328c-49c6-9020-ba775b72572b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.646858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dadff-097c-4966-9029-b0c8167c6db0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.652548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189485c-328c-49c6-9020-ba775b72572b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.659716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.661724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.661724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.662273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.663464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:27.664010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.667418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.667645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.667892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38fe8-424d-4154-b926-1c59b9faee4c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.671170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.671170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.673072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.676797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.705181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1065/2117] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:27.707583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.708208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.708832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.736265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1053/2118] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/projects => generated 385 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:27.737549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.757479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] MFA Rules not processed for user `b25f8f68987848d7864fd0b0119cf86a`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:27.759731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.822362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:27.831502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585ee50-d46d-4c6b-b992-dcc0ff505a0b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.839661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.842141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.842868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.842868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.844399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.850020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.860033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1066/2119] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:27.861303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.863400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:27.885037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c412a-4d60-450d-9af3-54f1dd066e6d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.891915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:27.893140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:27.893748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:27.894195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] PATH_INFO: `/v3/projects/53cde2850758461a80e892bc7cf85187` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:27.895714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] RBAC: Authorizing `identity:delete_project(project_id=53cde2850758461a80e892bc7cf85187)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:27.898065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:27.914355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1054/2120] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/projects => generated 399 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:27.914355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:27.984685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53cde2850758461a80e892bc7cf85187', 'request_id': 'req-b3543cde-781f-4f66-a7fe-11535189192e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:27.993363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3fd33-4b23-46a2-8af1-b605d4607427 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:27.993550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53cde2850758461a80e892bc7cf85187', 'request_id': 'req-b3543cde-781f-4f66-a7fe-11535189192e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.003886 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53cde2850758461a80e892bc7cf85187', 'request_id': 'req-b3543cde-781f-4f66-a7fe-11535189192e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.030649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] The token cache is being invalidate because project 53cde2850758461a80e892bc7cf85187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:28.030960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.032379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.035148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1067/2121] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:27 2022] DELETE /identity/v3/projects/53cde2850758461a80e892bc7cf85187 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:28.035751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.037376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.101343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.102072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.102365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.102679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.106274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.109134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.122762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:28.124370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543cde-781f-4f66-a7fe-11535189192e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.130691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.131619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.131995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.132385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.135492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.142271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1055/2122] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:27 2022] POST /identity/v3/users => generated 299 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:28.142271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.144841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.157709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8734d-9581-4a72-958d-4a4e16911b6b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.162694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.163368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.163662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.163949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.165279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.167267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.191522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1056/2123] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:28 2022] POST /identity/v3/users => generated 300 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:28.192415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.196332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.210271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7e1e7-9c30-4846-a0c2-dc9c36bba0d5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.210915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.212694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.215981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.217170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.217486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.217810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] PATH_INFO: `/v3/users/8896fde0a653429f9abaf727ecf187ea` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.219402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8896fde0a653429f9abaf727ecf187ea)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.221315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.223460 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1068/2124] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:28.224498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.226424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.245958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e1a1-5823-4f2a-951d-b645896d8020 service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.253771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.254550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.255096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.255568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.256411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.258178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.281483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1069/2125] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:28.282767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.298200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b137a-0846-47d0-9801-f9eebd4804a2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.306481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.308096 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.308485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.308874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.310814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.311497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8896fde0a653429f9abaf727ecf187ea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.313291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.331111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1070/2126] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:28.333344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.340455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:28.346471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8896fde0a653429f9abaf727ecf187ea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.351612 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1c3e8-8179-4266-9fb1-474b8eb05309 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.357873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8896fde0a653429f9abaf727ecf187ea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.364467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155e49-3425-4780-a958-154550f312fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.365405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155e49-3425-4780-a958-154550f312fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.365852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155e49-3425-4780-a958-154550f312fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.366212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155e49-3425-4780-a958-154550f312fc admin admin] PATH_INFO: `/v3/projects/dae490d39efd4a35940f426064b2b018/users/724d408d18d64048b832aa9288ae00ed/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.369859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8896fde0a653429f9abaf727ecf187ea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.370807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8896fde0a653429f9abaf727ecf187ea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:28.379041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1057/2127] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:28 2022] DELETE /identity/v3/users/8896fde0a653429f9abaf727ecf187ea => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:28.382979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.384817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.389865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155e49-3425-4780-a958-154550f312fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae490d39efd4a35940f426064b2b018, user_id=724d408d18d64048b832aa9288ae00ed, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.393715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155e49-3425-4780-a958-154550f312fc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.415708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1071/2128] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:28 2022] PUT /identity/v3/projects/dae490d39efd4a35940f426064b2b018/users/724d408d18d64048b832aa9288ae00ed/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:28.417128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.417521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.417837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.454861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:28.458817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.469374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144c1d4-904d-46ab-960e-69674a512490 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.553153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:28.563538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.565527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1072/2129] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:28.565698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.566703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.567490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.567850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.568027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.568632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.573093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.586313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1058/2130] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:28.591321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.596384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e643af0-02ba-49b1-97cb-5426b8213d8f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.602977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.603383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffdeec-6f80-4a2a-b592-e01065f634f8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.603648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.603942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.604226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.605145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.606841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.609770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbc13d-7218-494e-9d6f-67c63027a671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.610785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbc13d-7218-494e-9d6f-67c63027a671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.611250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbc13d-7218-494e-9d6f-67c63027a671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.611666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbc13d-7218-494e-9d6f-67c63027a671 admin admin] PATH_INFO: `/v3/projects/7ac9c87183db48a99048d0819ffad759/users/26e2abba38e34542823f1912fbb16f70/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.620470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1073/2131] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:28.626397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.645183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917dda2-2ccb-4f0b-ab3b-b7633452783f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.649250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbc13d-7218-494e-9d6f-67c63027a671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac9c87183db48a99048d0819ffad759, user_id=26e2abba38e34542823f1912fbb16f70, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.652010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbc13d-7218-494e-9d6f-67c63027a671 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.652799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.653950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.655541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.655541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] PATH_INFO: `/v3/projects/dae490d39efd4a35940f426064b2b018/users/724d408d18d64048b832aa9288ae00ed/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.669309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae490d39efd4a35940f426064b2b018, user_id=724d408d18d64048b832aa9288ae00ed, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.674032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.686877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1074/2132] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:28 2022] PUT /identity/v3/projects/dae490d39efd4a35940f426064b2b018/users/724d408d18d64048b832aa9288ae00ed/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:28.695675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.695914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1059/2133] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:28 2022] PUT /identity/v3/projects/7ac9c87183db48a99048d0819ffad759/users/26e2abba38e34542823f1912fbb16f70/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:28.696030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.698858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709fed-83d9-4f12-82da-bb14826d6b6e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.699166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709fed-83d9-4f12-82da-bb14826d6b6e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.699456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709fed-83d9-4f12-82da-bb14826d6b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.754630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-19709fed-83d9-4f12-82da-bb14826d6b6e None None] MFA Rules not processed for user `724d408d18d64048b832aa9288ae00ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:28.757027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-19709fed-83d9-4f12-82da-bb14826d6b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.816907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e681da5-f79a-4cb0-93ad-e3d0820afd9c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:28.824681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:28.825529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.826670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.827264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.887652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1060/2134] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:28.890103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:28.890403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:28.890694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:28.925799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:28.927421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:28.942689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] MFA Rules not processed for user `26e2abba38e34542823f1912fbb16f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:28.943729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:28.944268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:28.948067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1075/2135] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:44:28.948913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:28.954662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.140141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5872b-5701-4c6a-8040-fd422add3f97 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:29.149583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1061/2136] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:29.150620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:29.152299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:29.153197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:29.153703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:29.154528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] PATH_INFO: `/v3/projects/b4c402557b7946b69c364ac64e6a2e95` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:29.155532 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.155963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c402557b7946b69c364ac64e6a2e95)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:29.157929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:29.269553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c402557b7946b69c364ac64e6a2e95', 'request_id': 'req-159af98e-bc58-43b4-a99c-cb42bf6509aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:29.273750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:29.306561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94a4a3-30e3-4822-8876-d13911d61dee None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:29.307524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c402557b7946b69c364ac64e6a2e95', 'request_id': 'req-159af98e-bc58-43b4-a99c-cb42bf6509aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:29.322481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c402557b7946b69c364ac64e6a2e95', 'request_id': 'req-159af98e-bc58-43b4-a99c-cb42bf6509aa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:29.356165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] The token cache is being invalidate because project b4c402557b7946b69c364ac64e6a2e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:29.358604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:29.362827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:29.365518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1076/2137] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:28 2022] DELETE /identity/v3/projects/b4c402557b7946b69c364ac64e6a2e95 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:29.366425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:29.368238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.435102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:29.437088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:29.437562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:29.438013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:29.440856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.481418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159af98e-bc58-43b4-a99c-cb42bf6509aa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:29.489799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:29.490914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:29.491297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:29.491755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:29.493926 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.590194 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:29.598015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:29.601096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9c575-ec8a-4802-a370-d6b75e412ea4 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:29.624850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1062/2138] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:29.626335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:29.627922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4bdd8-5099-4786-a58d-e20b5af7b332 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:29.633492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:29.633930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:29.634794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:29.647685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1077/2139] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:29.675053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:29.676518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.838652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1063/2140] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:29.847479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:29.847875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:29.848253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:29.891291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:29.893712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:29.918821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1078/2141] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:29.920963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:29.937811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e005495-f48f-4783-8e7a-41fe6e921fa7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:29.956364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:29.957319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:29.957628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:29.957941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:29.960267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:29.960602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:29.968257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:30.033871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1064/2142] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:30.038193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:30.055493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679ee91-15b7-4d1c-9799-49df9570f8f2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:30.070938 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:30.072437 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:30.075538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:30.076756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:30.080046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:30.086860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:30.196863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1079/2143] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:30 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:30.201686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:30.222115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fe399-912d-4fa5-8e36-8e99a5bc9e98 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:30.330662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:30.387484 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:30.389094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:30.389496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:30.389955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:30.390806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:30.396629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:30.518107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1065/2144] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:30 2022] POST /identity/v3/users => generated 286 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:30.523041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:30.539453 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01624d-16e1-4f01-8e68-4b08638e1e6e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:30.585196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:30.585920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:30.586207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:30.586956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:30.590627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:30.598733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:30.636289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:30.636768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1080/2145] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:30.651453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb179e-b137-4c1a-8a2a-ae82eed83f3b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:30.666115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d58b63-073a-47dc-b125-6b7fa3b1d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:30.672184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58b63-073a-47dc-b125-6b7fa3b1d36c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:30.672184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58b63-073a-47dc-b125-6b7fa3b1d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:30.672184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58b63-073a-47dc-b125-6b7fa3b1d36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:30.672184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d58b63-073a-47dc-b125-6b7fa3b1d36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:30.672184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d58b63-073a-47dc-b125-6b7fa3b1d36c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:30.707115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1066/2146] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:30.720114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:30.750730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f4dea-1382-4ef8-ace7-ce5c578e7eee admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:30.770934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:30.771863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:30.772149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:30.772430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] PATH_INFO: `/v3/projects/95ad99f5649a4af8965d258b748ad428/users/0db68ec309c44989a2b9d373da3294c5/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:30.799266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ad99f5649a4af8965d258b748ad428, user_id=0db68ec309c44989a2b9d373da3294c5, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:30.823686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:30.850174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1081/2147] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:30 2022] PUT /identity/v3/projects/95ad99f5649a4af8965d258b748ad428/users/0db68ec309c44989a2b9d373da3294c5/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:30.862169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:30.862684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:30.863111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:30.915647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] MFA Rules not processed for user `0db68ec309c44989a2b9d373da3294c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:30.915647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:31.015295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1067/2148] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:31.030147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:31.033082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:31.066929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:31.068662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:31.141179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fe3c3-15b8-4c0b-a45e-0b43b7b5e59a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:31.149033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad18aec-2052-443e-86e6-5d7089161c13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:31.150743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18aec-2052-443e-86e6-5d7089161c13 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:31.152682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18aec-2052-443e-86e6-5d7089161c13 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:31.153052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18aec-2052-443e-86e6-5d7089161c13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:31.165627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad18aec-2052-443e-86e6-5d7089161c13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:31.168554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad18aec-2052-443e-86e6-5d7089161c13 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:31.183774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1082/2149] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:31.197010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7253d-82e0-4b8e-86f2-8ba8b2a050cd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:31.208118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:31.208118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:31.208118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:31.208118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] PATH_INFO: `/v3/users/30902ec0e5594cca8dff8a7113d5e39a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:31.214574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30902ec0e5594cca8dff8a7113d5e39a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:31.218978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:31.301131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30902ec0e5594cca8dff8a7113d5e39a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:31.304986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:31.338597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30902ec0e5594cca8dff8a7113d5e39a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:31.354465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30902ec0e5594cca8dff8a7113d5e39a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:31.376091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30902ec0e5594cca8dff8a7113d5e39a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:31.377124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30902ec0e5594cca8dff8a7113d5e39a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:31.389272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1068/2150] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:31 2022] DELETE /identity/v3/users/30902ec0e5594cca8dff8a7113d5e39a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:31.391073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:31.392023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:31.392023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:31.420526 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:31.422177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:31.582881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1083/2151] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:31.593618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:31.595113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:31.756639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66923ad-8b3b-4273-bb54-df5587a07584 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:31.770055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:31.771383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:31.771730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:31.772026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:31.801336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:31.805043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:31.817550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1069/2152] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:32.472558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:32.474296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:32.511697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f52f89-7aae-40d9-b7ae-7cc25566a404 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:32.525989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:32.527269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:32.527758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:32.528198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] PATH_INFO: `/v3/projects/0e8c0eb611504140a00f13e7cedaf70a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:32.529872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8c0eb611504140a00f13e7cedaf70a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:32.532193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:32.575010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:32.627574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8c0eb611504140a00f13e7cedaf70a', 'request_id': 'req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.636184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8c0eb611504140a00f13e7cedaf70a', 'request_id': 'req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.646307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8c0eb611504140a00f13e7cedaf70a', 'request_id': 'req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.665779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] The token cache is being invalidate because project 0e8c0eb611504140a00f13e7cedaf70a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:32.666113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.667143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09a6ce9c-fc07-400d-bc7f-fc4d88ef5ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.670778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1084/2153] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:32 2022] DELETE /identity/v3/projects/0e8c0eb611504140a00f13e7cedaf70a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:32.674985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:32.686100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e6a2b-b0b7-46b0-84a7-511db30a36e2 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:32.785265 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:32.786598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:32.787177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:32.787692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] PATH_INFO: `/v3/users/724d408d18d64048b832aa9288ae00ed` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:32.789486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724d408d18d64048b832aa9288ae00ed)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:32.791833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:32.886792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724d408d18d64048b832aa9288ae00ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.921030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724d408d18d64048b832aa9288ae00ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.934282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:32.949984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724d408d18d64048b832aa9288ae00ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.967502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724d408d18d64048b832aa9288ae00ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.968209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a461bfb3-cd50-43c7-b431-904b2aa957a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724d408d18d64048b832aa9288ae00ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:32.982976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1070/2154] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:32 2022] DELETE /identity/v3/users/724d408d18d64048b832aa9288ae00ed => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:33.156326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699fb1-8782-455d-b407-f2cab0e5e0d6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.156765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699fb1-8782-455d-b407-f2cab0e5e0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.157058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699fb1-8782-455d-b407-f2cab0e5e0d6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.158732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1085/2155] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:44:33.171744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.172470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.172873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.199814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] MFA Rules not processed for user `fddfa14cf721489595e79ffaafef6322`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:33.202073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.315550 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1071/2156] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:33.334486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699fb1-8782-455d-b407-f2cab0e5e0d6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:33.337958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-99699fb1-8782-455d-b407-f2cab0e5e0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.352965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:33.354666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.391147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0571c-f067-4a73-8df5-c6c1357e65a1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:33.396599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:33.397278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.397594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.397918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.410373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:33.412369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:33.427649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1072/2157] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:33.518941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:33.521111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699fb1-8782-455d-b407-f2cab0e5e0d6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:33.521888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.530465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:33.531632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.532321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.532772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] PATH_INFO: `/v3/users/05add5effa924b908fc522cb16f8387c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.534063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=05add5effa924b908fc522cb16f8387c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:33.537495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:33.553439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd08a0-bfd3-49df-9f9b-6192c3750f01 service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:33.558578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:33.559623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.559974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.560476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] PATH_INFO: `/v3/projects/dae490d39efd4a35940f426064b2b018` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.561859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae490d39efd4a35940f426064b2b018)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:33.563961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:33.611046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05add5effa924b908fc522cb16f8387c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.611866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05add5effa924b908fc522cb16f8387c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.626117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05add5effa924b908fc522cb16f8387c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.627577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05add5effa924b908fc522cb16f8387c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.636967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae490d39efd4a35940f426064b2b018', 'request_id': 'req-13025ce8-faae-4b82-81a3-cff0cd5d2452'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.640961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1471f337-84d9-4fbd-ad34-69bbc4783e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05add5effa924b908fc522cb16f8387c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.644627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae490d39efd4a35940f426064b2b018', 'request_id': 'req-13025ce8-faae-4b82-81a3-cff0cd5d2452'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.656305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae490d39efd4a35940f426064b2b018', 'request_id': 'req-13025ce8-faae-4b82-81a3-cff0cd5d2452'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.682099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] The token cache is being invalidate because project dae490d39efd4a35940f426064b2b018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:33.683127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.683618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-13025ce8-faae-4b82-81a3-cff0cd5d2452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:33.686083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1073/2158] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:33 2022] DELETE /identity/v3/projects/dae490d39efd4a35940f426064b2b018 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:33.692375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.692748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.693163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.696810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1086/2159] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:33 2022] DELETE /identity/v3/users/05add5effa924b908fc522cb16f8387c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:33.716186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:33.717866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.781273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:33.799380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1074/2160] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:33.807176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.807750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.808340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.835326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:33.836563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.848930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1087/2161] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:33.853690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:33.865982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262df96-70ab-4fbd-ad86-1af12f44f19d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:33.870912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:33.871654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.871953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.872234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.873026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:33.873348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:33.875452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:33.892131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1075/2162] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:33.892481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:33.902866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de768c-f5da-45bc-a899-9d96c22dd11d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:33.912781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:33.913428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:33.913736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:33.914018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:33.914718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:33.916653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:33.941791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:33.943607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:33.954606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:33.965686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5e558-b6dd-4970-9533-4620146f1ff0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:33.967422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1088/2163] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:33 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:33.971243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.049431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.050719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.051180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.051763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] PATH_INFO: `/v3/projects/3f4caa81d58a4daf90980e624d1c2526` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.053232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f4caa81d58a4daf90980e624d1c2526)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.056183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.060133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4515b-5267-4631-a4af-24ad1e7ef065 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.067270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.068945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.068945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.069702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.070057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.072511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.098151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1089/2164] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:33 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:34.102159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.113709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d056ae2-4b82-468e-97c0-6d12a6a0d890 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.135798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f4caa81d58a4daf90980e624d1c2526', 'request_id': 'req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:34.150084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f4caa81d58a4daf90980e624d1c2526', 'request_id': 'req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:34.157591 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f4caa81d58a4daf90980e624d1c2526', 'request_id': 'req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:34.175890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] The token cache is being invalidate because project 3f4caa81d58a4daf90980e624d1c2526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:34.176230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:34.176969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a9b237d1-c2f6-421b-b9d6-68bb0898a9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:34.179602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1076/2165] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:33 2022] DELETE /identity/v3/projects/3f4caa81d58a4daf90980e624d1c2526 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:34.185945 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7c4a-8c00-4da0-b0a5-d21848d9cd94 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.186170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7c4a-8c00-4da0-b0a5-d21848d9cd94 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.186426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7c4a-8c00-4da0-b0a5-d21848d9cd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.214170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-aaad7c4a-8c00-4da0-b0a5-d21848d9cd94 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:34.216073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aaad7c4a-8c00-4da0-b0a5-d21848d9cd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:34.217208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.217972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.218283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.218568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.219319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.221163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.228895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1077/2166] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:34.233470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1090/2167] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:34.239287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.239588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.239877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.245457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.253045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:34.268614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:34.269896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:34.277448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99758-fe21-4b0a-b912-81b4064845e4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.282460 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1078/2168] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:34.283711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.284398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.284692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.284981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.285859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.286579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.287913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.297763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9684973-10a5-4989-9302-d61db3162be4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.308045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1091/2169] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:34.312239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.322515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.323954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.324342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.324648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.325523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:34.326598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.328030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.328753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045438c9-900a-47aa-b83d-3d5e1be4a6b0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.345922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678b963-9f4b-4bf5-b4bc-85ff5a658a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.347131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b963-9f4b-4bf5-b4bc-85ff5a658a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.347421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b963-9f4b-4bf5-b4bc-85ff5a658a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.347702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b963-9f4b-4bf5-b4bc-85ff5a658a4e admin admin] PATH_INFO: `/v3/projects/8ca143f7843c452685c27a6cb1e67307/users/208fd2b34f7040189f0195a09eb7f721/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.371806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1079/2170] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:44:34.378669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.385320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678b963-9f4b-4bf5-b4bc-85ff5a658a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca143f7843c452685c27a6cb1e67307, user_id=208fd2b34f7040189f0195a09eb7f721, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.391154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678b963-9f4b-4bf5-b4bc-85ff5a658a4e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.393343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81730f5-fe5d-44ce-8287-b26c3c05239e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.421734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1092/2171] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:34 2022] PUT /identity/v3/projects/8ca143f7843c452685c27a6cb1e67307/users/208fd2b34f7040189f0195a09eb7f721/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:34.426658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.426847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.427129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.491238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:34.492866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:34.500374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:34.528889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.530764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.531846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.532140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.532857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.534867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.542707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:34.565728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1093/2172] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:34.579405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.581303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:34.583228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1080/2173] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:34 2022] POST /identity/v3/projects => generated 355 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:34.584194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.668552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e73911-ee30-4fd6-8c15-cb4601041de1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.668974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe404bc-bba2-400e-b02f-2e3c3753b080 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.678024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.679000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.679452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.679908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.681570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.684043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.715736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1081/2174] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:34 2022] POST /identity/v3/users => generated 278 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:34.722347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.736972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c05584-eaa0-4426-8e04-4e25252efe10 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.742910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.743853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.743997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.744439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.744711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.745059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.745177 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.745343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.746181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.748567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.756626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.758323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.761322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1082/2175] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:34.766669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.777049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1094/2176] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:34.783141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9cd18-ccd5-45cb-933e-5bba3e5bb183 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.789609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705b7b3-6677-4cf9-aa54-92c36852851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.790371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705b7b3-6677-4cf9-aa54-92c36852851c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.790662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705b7b3-6677-4cf9-aa54-92c36852851c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.790999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705b7b3-6677-4cf9-aa54-92c36852851c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.791748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705b7b3-6677-4cf9-aa54-92c36852851c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.794039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705b7b3-6677-4cf9-aa54-92c36852851c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.808049 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1083/2177] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:34.812800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:34.826019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2af24-e056-4554-86b0-9d0207ea264b service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:34.836502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:34.842983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.843203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.843454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] PATH_INFO: `/v3/projects/49da3e7f6a8943d697b71f33cbe0c08d/users/0273e65563454155ada060815b8cfcfa/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.870761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49da3e7f6a8943d697b71f33cbe0c08d, user_id=0273e65563454155ada060815b8cfcfa, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:34.876368 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:34.912089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1095/2178] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:34 2022] PUT /identity/v3/projects/49da3e7f6a8943d697b71f33cbe0c08d/users/0273e65563454155ada060815b8cfcfa/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:34.914889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:34.915329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:34.915620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:34.969911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] MFA Rules not processed for user `0273e65563454155ada060815b8cfcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:34.976428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:35.068283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1084/2179] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:34 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:35.072423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.167439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e382-6e41-4af0-a80b-98bda93f1be5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.173737 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.174434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.174721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.175012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.175869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.177835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.228922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1096/2180] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/projects => generated 355 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:35.229339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.250326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be362a3-8b9b-4fbd-bbd3-8c236c9cde36 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.330569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:35.405266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.406331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.406631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.406959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.408199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.410380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.484434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1085/2181] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/users => generated 284 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:35.488092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.502493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a53a9-9147-48df-9243-9460cfce1882 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.511779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a67f-123c-456e-888e-c6a486949d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.512581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a67f-123c-456e-888e-c6a486949d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.513006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a67f-123c-456e-888e-c6a486949d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.514004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a67f-123c-456e-888e-c6a486949d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.514870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a67f-123c-456e-888e-c6a486949d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.516417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a67f-123c-456e-888e-c6a486949d09 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.518878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:35.532205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1097/2182] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:35.533686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.547857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905508-fe0b-44f3-a528-058ae9eef986 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.554478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.555455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.555751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.556035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] PATH_INFO: `/v3/projects/94130d322fb34d7caeae96bc46657b7d/users/160e12c4f37d4cbb893265c8e02524cd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.603283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94130d322fb34d7caeae96bc46657b7d, user_id=160e12c4f37d4cbb893265c8e02524cd, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.614015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.617981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.618511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.618695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.641159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1086/2183] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:35 2022] PUT /identity/v3/projects/94130d322fb34d7caeae96bc46657b7d/users/160e12c4f37d4cbb893265c8e02524cd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:35.646005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.665022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] MFA Rules not processed for user `208fd2b34f7040189f0195a09eb7f721`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:35.666952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:35.757181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8d86e-6a80-4b8d-8d08-3ab9396a33d3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.766467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.767283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.767642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.767992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.768820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.770564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.779532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1098/2184] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:35.784051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.791755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1087/2185] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:35.793396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.801258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa1b38-a3e1-4672-992c-788c021306e6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.807900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59260454-04a1-449a-bef8-f7ac384f6ac6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.813712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.814718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.815112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ebc22-45d2-46a1-b637-b4af4af7ea27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.815371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.815943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.816475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ebc22-45d2-46a1-b637-b4af4af7ea27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.816833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ebc22-45d2-46a1-b637-b4af4af7ea27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.817184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ebc22-45d2-46a1-b637-b4af4af7ea27 admin admin] PATH_INFO: `/v3/projects/94130d322fb34d7caeae96bc46657b7d/users/160e12c4f37d4cbb893265c8e02524cd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.819362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ebc22-45d2-46a1-b637-b4af4af7ea27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94130d322fb34d7caeae96bc46657b7d, user_id=160e12c4f37d4cbb893265c8e02524cd, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.819589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.822183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.822318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ebc22-45d2-46a1-b637-b4af4af7ea27 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.838986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1088/2186] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:35 2022] PUT /identity/v3/projects/94130d322fb34d7caeae96bc46657b7d/users/160e12c4f37d4cbb893265c8e02524cd/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:35.851178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.851746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.852151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.869343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1099/2187] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:35.873281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.886027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] MFA Rules not processed for user `160e12c4f37d4cbb893265c8e02524cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:35.887314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:35.968035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c72ec-958e-45b5-bda9-93fc475ca7d7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:35.973377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:35.974094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:35.974385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:35.974672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:35.975268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:35.976284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:35.978878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:35.981805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1089/2188] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:35.996132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:35.998507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:36.007582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1100/2189] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:35 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:36.008969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.020758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d996-673e-4113-b6dd-e781ce1bf6e6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.027299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.027984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.028272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.028553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.030061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.032391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.045764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1101/2190] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:36.049530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.059686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd950-1c39-4dd2-af48-434b6083c45b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.066411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.067829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.067829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.068383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] PATH_INFO: `/v3/projects/903da9bc4bd7448e8f7fad37bf4c8b4b/users/df77b8c760b643faba0b9edc36345217/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.079303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7c6e8-85a0-4462-b988-8b07dada99ec None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.084647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.086658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.087995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.088348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.099829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=903da9bc4bd7448e8f7fad37bf4c8b4b, user_id=df77b8c760b643faba0b9edc36345217, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.101966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.102797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.103590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.116581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1090/2191] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:36.121571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1102/2192] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:36 2022] PUT /identity/v3/projects/903da9bc4bd7448e8f7fad37bf4c8b4b/users/df77b8c760b643faba0b9edc36345217/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:36.126932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.140967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9312-f241-40b2-be43-3d147904f6f5 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.209328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.241367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.242377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.242843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.243323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.244262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.244414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:36.246278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.261707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1091/2193] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:36.266278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:36.280911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e03bb-e035-4b3f-b887-c0dc98ffff81 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.287742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fa65e-f16b-4492-a654-e29e802d495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.289071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fa65e-f16b-4492-a654-e29e802d495b admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.289427 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fa65e-f16b-4492-a654-e29e802d495b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.289946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fa65e-f16b-4492-a654-e29e802d495b admin admin] PATH_INFO: `/v3/projects/903da9bc4bd7448e8f7fad37bf4c8b4b/users/df77b8c760b643faba0b9edc36345217/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.292838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fa65e-f16b-4492-a654-e29e802d495b admin admin] RBAC: Authorizing `identity:create_grant(project_id=903da9bc4bd7448e8f7fad37bf4c8b4b, user_id=df77b8c760b643faba0b9edc36345217, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.296919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fa65e-f16b-4492-a654-e29e802d495b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.310200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49b50e-8d6b-489a-8eb9-19505f5aaa0a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:36.316124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1092/2194] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:36 2022] PUT /identity/v3/projects/903da9bc4bd7448e8f7fad37bf4c8b4b/users/df77b8c760b643faba0b9edc36345217/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:36.391343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:36.392286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:36.392634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:36.393032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:36.496778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:36.500413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:36.525290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1103/2195] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:37.246149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:37.246661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:37.246759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:37.287772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] MFA Rules not processed for user `df77b8c760b643faba0b9edc36345217`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:37.290388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:37.485027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1093/2196] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:37.521554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:37.538275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d19f7-e8ab-4164-83c7-b63bbd913bed service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:37.591722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:37.592720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:37.593188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:37.593654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:37.727488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:37.729905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:37.744893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1104/2197] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:39.371493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:39.392169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607c43b-e8be-48f2-9163-8cac384903f6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:39.396462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a75e83-5dec-4c19-a35d-9878d85cfe9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:39.397130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a75e83-5dec-4c19-a35d-9878d85cfe9d service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:39.397432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a75e83-5dec-4c19-a35d-9878d85cfe9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:39.397726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a75e83-5dec-4c19-a35d-9878d85cfe9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:39.411005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a75e83-5dec-4c19-a35d-9878d85cfe9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:39.413210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a75e83-5dec-4c19-a35d-9878d85cfe9d service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:39.422087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:39.434043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1094/2198] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:40.020311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.021545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.063376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:40.104899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f2c68-cba2-4359-9d92-350d70daf258 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.110614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.111515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.111804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.112085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] PATH_INFO: `/v3/users/26e2abba38e34542823f1912fbb16f70` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.113322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e2abba38e34542823f1912fbb16f70)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.115359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.169963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e2abba38e34542823f1912fbb16f70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.172215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e2abba38e34542823f1912fbb16f70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.181882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e2abba38e34542823f1912fbb16f70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.182618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e2abba38e34542823f1912fbb16f70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.191071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e2abba38e34542823f1912fbb16f70'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:40.216144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1105/2199] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:40 2022] DELETE /identity/v3/users/26e2abba38e34542823f1912fbb16f70 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:40.221298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.221636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.222022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.244012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:40.245277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.318209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1095/2200] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:40.324870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.326866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.393296 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080428a5-75d8-458e-bbfe-39b902e850f3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:40.397191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ee040-7752-49a8-93f7-2a403862fb32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:40.397885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ee040-7752-49a8-93f7-2a403862fb32 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:40.398203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ee040-7752-49a8-93f7-2a403862fb32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:40.398460 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ee040-7752-49a8-93f7-2a403862fb32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:40.411573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ee040-7752-49a8-93f7-2a403862fb32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:40.413220 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ee040-7752-49a8-93f7-2a403862fb32 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:40.425711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1106/2201] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:40.915886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:40.925606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:40.968038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9270db-d8bf-447b-a0f5-6a0ca9cae0c2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:41.014283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:41.016085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:41.016955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:41.021114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] PATH_INFO: `/v3/projects/7ac9c87183db48a99048d0819ffad759` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:41.031128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac9c87183db48a99048d0819ffad759)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:41.036923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:41.173702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac9c87183db48a99048d0819ffad759', 'request_id': 'req-51947168-d98d-4fc1-9ff9-9bb58db40b30'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:41.181566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac9c87183db48a99048d0819ffad759', 'request_id': 'req-51947168-d98d-4fc1-9ff9-9bb58db40b30'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:41.191024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac9c87183db48a99048d0819ffad759', 'request_id': 'req-51947168-d98d-4fc1-9ff9-9bb58db40b30'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:41.215756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] The token cache is being invalidate because project 7ac9c87183db48a99048d0819ffad759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:41.217358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:41.218122 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-51947168-d98d-4fc1-9ff9-9bb58db40b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:41.223880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1096/2202] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:40 2022] DELETE /identity/v3/projects/7ac9c87183db48a99048d0819ffad759 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:42.214383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:42.214956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:42.215086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:42.232957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] MFA Rules not processed for user `163fb6502e0442c7aefc0c122ed4c035`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:42.234287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:42.280801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:42.295452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1107/2203] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:44.651222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.651821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.651821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.671943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:44.673303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:44.751978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1097/2204] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:44.767706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:44.769556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:44.842427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab4432-76ed-489b-a21f-d59bc98113cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:44.847632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:44.848260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:44.848551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:44.848830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:44.858897 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:44.860442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:44.869943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1108/2205] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:44:53.859680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:53.863099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:53.918734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:53.969696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df592b-a091-40c8-a0ab-a7dafc77ff05 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:53.976130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:53.977036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:53.977471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:53.977813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] PATH_INFO: `/v3/users/208fd2b34f7040189f0195a09eb7f721` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:53.979707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=208fd2b34f7040189f0195a09eb7f721)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:53.986388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:54.060788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208fd2b34f7040189f0195a09eb7f721'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.080779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208fd2b34f7040189f0195a09eb7f721'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.092417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208fd2b34f7040189f0195a09eb7f721'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.102919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208fd2b34f7040189f0195a09eb7f721'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.103563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208fd2b34f7040189f0195a09eb7f721'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.115588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1098/2206] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:53 2022] DELETE /identity/v3/users/208fd2b34f7040189f0195a09eb7f721 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:54.135271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:54.138370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:54.156852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963766-cd95-49a0-af2e-32031d1c64c3 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:54.246140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:54.247381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:54.248048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:54.248669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] PATH_INFO: `/v3/users/0273e65563454155ada060815b8cfcfa` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:54.250523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0273e65563454155ada060815b8cfcfa)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:54.253069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:54.342997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0273e65563454155ada060815b8cfcfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.344042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0273e65563454155ada060815b8cfcfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.352808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0273e65563454155ada060815b8cfcfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.353820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0273e65563454155ada060815b8cfcfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.364528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-41317372-947b-48d6-9939-f12d6030f002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0273e65563454155ada060815b8cfcfa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.399232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:54.402188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:54.428382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10407b-da77-436d-8b0b-48c4aa520f3f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:54.431678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1109/2207] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:54 2022] DELETE /identity/v3/users/0273e65563454155ada060815b8cfcfa => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:54.433916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:54.434218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:54.434502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:54.442872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:54.455161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:54.458421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:54.520598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:54.521873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:54.522161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:54.522443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] PATH_INFO: `/v3/projects/8ca143f7843c452685c27a6cb1e67307` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:54.523750 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca143f7843c452685c27a6cb1e67307)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:54.525784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:54.530280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1110/2208] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:54.546558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:54.551880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:54.603546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:54.608580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca143f7843c452685c27a6cb1e67307', 'request_id': 'req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.618777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca143f7843c452685c27a6cb1e67307', 'request_id': 'req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.622316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3cc6-58a6-416f-a82d-94b2e6854743 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:54.630110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca143f7843c452685c27a6cb1e67307', 'request_id': 'req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.652029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] The token cache is being invalidate because project 8ca143f7843c452685c27a6cb1e67307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:54.652393 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.653085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.657941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1099/2209] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:54 2022] DELETE /identity/v3/projects/8ca143f7843c452685c27a6cb1e67307 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:54.659811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:54.663840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:54.692471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:54.696429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:54.696792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:54.697149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:54.699860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:54.774954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcbc43-ff26-40ed-a9f7-ea06c2585e68 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:54.779765 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:54.780719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:54.781075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:54.781364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] PATH_INFO: `/v3/users/df77b8c760b643faba0b9edc36345217` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:54.782626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df77b8c760b643faba0b9edc36345217)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:54.784432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:54.785195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:54.786208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:54.821821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1111/2210] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:54.854736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df77b8c760b643faba0b9edc36345217'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.875159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df77b8c760b643faba0b9edc36345217'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.886169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df77b8c760b643faba0b9edc36345217'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.895449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df77b8c760b643faba0b9edc36345217'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.896547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df77b8c760b643faba0b9edc36345217'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:54.908407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1100/2211] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:54 2022] DELETE /identity/v3/users/df77b8c760b643faba0b9edc36345217 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:55.183117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.196548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c5541-8194-4b83-86bb-8e4ff550c0c5 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.203988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.209388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.212272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:55.306152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.307678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.308076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.308462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] PATH_INFO: `/v3/projects/903da9bc4bd7448e8f7fad37bf4c8b4b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.309951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903da9bc4bd7448e8f7fad37bf4c8b4b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.312100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.322617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:55.331482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df662e5a-d9aa-4b22-aa9d-478217d382e9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.342846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.343801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.344089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.344598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] PATH_INFO: `/v3/projects/49da3e7f6a8943d697b71f33cbe0c08d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.345731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49da3e7f6a8943d697b71f33cbe0c08d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.348436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.377753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903da9bc4bd7448e8f7fad37bf4c8b4b', 'request_id': 'req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.390134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903da9bc4bd7448e8f7fad37bf4c8b4b', 'request_id': 'req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.402290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903da9bc4bd7448e8f7fad37bf4c8b4b', 'request_id': 'req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.422813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] The token cache is being invalidate because project 903da9bc4bd7448e8f7fad37bf4c8b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:55.423113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.424157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a29e4c8-cb75-4894-98b2-3c5e4c87e565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.431431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1112/2212] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:55 2022] DELETE /identity/v3/projects/903da9bc4bd7448e8f7fad37bf4c8b4b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:55.434135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e45d9-e980-4847-9553-0574d385525a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.434558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e45d9-e980-4847-9553-0574d385525a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.435150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e45d9-e980-4847-9553-0574d385525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.445177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49da3e7f6a8943d697b71f33cbe0c08d', 'request_id': 'req-0517ebed-d144-4b3e-b1b5-bf84f0091964'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.452911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49da3e7f6a8943d697b71f33cbe0c08d', 'request_id': 'req-0517ebed-d144-4b3e-b1b5-bf84f0091964'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.470146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a23e45d9-e980-4847-9553-0574d385525a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:55.472498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a23e45d9-e980-4847-9553-0574d385525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.490413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49da3e7f6a8943d697b71f33cbe0c08d', 'request_id': 'req-0517ebed-d144-4b3e-b1b5-bf84f0091964'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.511827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] The token cache is being invalidate because project 49da3e7f6a8943d697b71f33cbe0c08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:55.512651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.514224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.516855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1101/2213] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:55 2022] DELETE /identity/v3/projects/49da3e7f6a8943d697b71f33cbe0c08d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:55.521635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.527094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.573783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1113/2214] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:55.576777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.577056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.577353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.599507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:55.600758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.613678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1114/2215] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:55.614747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.624402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc784b3-a52c-42fa-ae04-515fc5f72b0f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.630374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.631051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.631340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.631620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.631852 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ebed-d144-4b3e-b1b5-bf84f0091964 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.632362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:55.632676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.634344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.636862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.637861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.638215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.638564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] PATH_INFO: `/v3/users/160e12c4f37d4cbb893265c8e02524cd` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.639853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=160e12c4f37d4cbb893265c8e02524cd)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.641862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.647804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1115/2216] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:55.652621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.664393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40358808-d2fa-4363-91f7-867639ac92ea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.670977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.671736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.672620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.672913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.673721 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.676501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.707424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:55.719655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1116/2217] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:55 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:55.719963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160e12c4f37d4cbb893265c8e02524cd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.721898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.734794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160e12c4f37d4cbb893265c8e02524cd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.751681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160e12c4f37d4cbb893265c8e02524cd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.762295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160e12c4f37d4cbb893265c8e02524cd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.763012 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160e12c4f37d4cbb893265c8e02524cd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:55.770885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1102/2218] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:55 2022] DELETE /identity/v3/users/160e12c4f37d4cbb893265c8e02524cd => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:55.799284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1f6c-ba57-4cb4-9c72-b6bde44b39b5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.880202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.880821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.881115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.881400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.882148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.884081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.928458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:55.929879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1117/2219] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:55 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:55.930762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:55.941003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:55.945598 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23a6ad-a8d6-48b5-83e9-cd5848a43447 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:55.951540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:55.952885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:55.953225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:55.953824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:55.955458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:55.957885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:55.978859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1103/2220] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:55.982969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.001332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc72c6-65de-4bdd-b9c4-02063ed9d2cf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.007405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6454a8-96f5-4e8f-86a8-cc411b625c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.008080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6454a8-96f5-4e8f-86a8-cc411b625c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.008367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6454a8-96f5-4e8f-86a8-cc411b625c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.008754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6454a8-96f5-4e8f-86a8-cc411b625c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.009549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6454a8-96f5-4e8f-86a8-cc411b625c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.011579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6454a8-96f5-4e8f-86a8-cc411b625c11 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.043734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1118/2221] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:44:56.045111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.056488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42223a20-aa57-4d1b-8324-58ddfad20bc8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.061756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.062668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.062965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.063247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] PATH_INFO: `/v3/projects/b5044c2755de4142a44628c1d8058537/users/16698e1bead8473095b34a078018e625/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.087266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5044c2755de4142a44628c1d8058537, user_id=16698e1bead8473095b34a078018e625, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.090560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.116620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1104/2222] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:56 2022] PUT /identity/v3/projects/b5044c2755de4142a44628c1d8058537/users/16698e1bead8473095b34a078018e625/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:56.127054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.127542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.127899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.161990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] MFA Rules not processed for user `16698e1bead8473095b34a078018e625`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:56.163226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.194031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.195877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.253723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1119/2223] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:56.263539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.266452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.285248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f277-fd45-4823-a2cd-e9fd85b5f09e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.292464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.293770 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.294423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.294742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] PATH_INFO: `/v3/projects/94130d322fb34d7caeae96bc46657b7d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.296087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94130d322fb34d7caeae96bc46657b7d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.298177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.300647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:56.353913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66700989-9188-4237-be37-4f8aa944a0dc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.355056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94130d322fb34d7caeae96bc46657b7d', 'request_id': 'req-22090940-808a-4d97-a0cf-efa111611ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.361111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94130d322fb34d7caeae96bc46657b7d', 'request_id': 'req-22090940-808a-4d97-a0cf-efa111611ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.368413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94130d322fb34d7caeae96bc46657b7d', 'request_id': 'req-22090940-808a-4d97-a0cf-efa111611ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.382000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] The token cache is being invalidate because project 94130d322fb34d7caeae96bc46657b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:44:56.382529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.385497 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-22090940-808a-4d97-a0cf-efa111611ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:56.387624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1105/2224] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:44:56 2022] DELETE /identity/v3/projects/94130d322fb34d7caeae96bc46657b7d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:56.394086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.394450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.394808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.410404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.411298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.411976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.412443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.414402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.418756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:56.419946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.483622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.485111 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67025e61-e499-42cb-a06f-b0e2ce7aaf02 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.489902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:56.496092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1120/2225] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:44:56.506462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1106/2226] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:44:56.513689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.513994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.514283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.532890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:56.534844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:56.543495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1121/2227] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:56.548142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.573724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41965486-3699-46f4-be19-bc6ff5117418 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.576683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.577417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.577810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.578164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.578907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:44:56.579218 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.581526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.594491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1107/2228] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:44:56.598882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.609859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb56c9-dc58-4044-bee0-865e9e3cb364 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.634822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.636062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.636544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.637141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.638207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.641436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.683536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1122/2229] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:56.687652 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.700364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff5102-490d-41ba-838e-47f5fec60fb3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.817271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.818219 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.820462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.821896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.823487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.826807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.855071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1108/2230] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:56 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:44:56.859208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.883558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e04dac-5818-4c26-ad69-b163919e59ba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.900635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.901317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.904031 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.904381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.905150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.906824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.921121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1123/2231] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:56.930438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.943755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160daab2-4bfa-4d26-892d-782c27419d31 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.949290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a0cb0-c98a-4ab3-bc16-2c9dcf4ffaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.950259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a0cb0-c98a-4ab3-bc16-2c9dcf4ffaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.950553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a0cb0-c98a-4ab3-bc16-2c9dcf4ffaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.950839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a0cb0-c98a-4ab3-bc16-2c9dcf4ffaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:56.951646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a0cb0-c98a-4ab3-bc16-2c9dcf4ffaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:56.954498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a0cb0-c98a-4ab3-bc16-2c9dcf4ffaf6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:56.976056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1109/2232] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:56.980357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:56.991464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edad72-f6c8-4bef-a98e-f02ec508b7ee admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:56.996207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:56.997088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:56.997381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:56.997790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] PATH_INFO: `/v3/projects/20ec87de09114441bdc57f3cae2b8e3d/users/0ba5fb7d7b4e41f9a34a3b02119e42e4/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.018635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=20ec87de09114441bdc57f3cae2b8e3d, user_id=0ba5fb7d7b4e41f9a34a3b02119e42e4, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.021507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.039823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1124/2233] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:56 2022] PUT /identity/v3/projects/20ec87de09114441bdc57f3cae2b8e3d/users/0ba5fb7d7b4e41f9a34a3b02119e42e4/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:57.052090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.052400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.052772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.083338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] MFA Rules not processed for user `0ba5fb7d7b4e41f9a34a3b02119e42e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:57.085029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:57.180900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1110/2234] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:57.185426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.220063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:57.266506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082aed4-356f-4347-a612-68b1f5353dbf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.272045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.272861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.273260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.273703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.274508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.277542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.315662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1125/2235] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:44:57 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:57.316914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.328199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37388080-76df-404d-a4bc-e11988f7fcb4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.379944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:57.389812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.390838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.391199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.391556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.392380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.395295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.416641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1111/2236] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:44:57 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:44:57.420202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.431249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac8fcc-2b4f-4fe8-a177-aa40e291a7c6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.435855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.436530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.436853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.437107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.437867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.439479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.465053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1126/2237] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:57.468658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.479221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4243e-6e83-469e-896c-32c78d5b300b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.483987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.484893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.485181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.485465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] PATH_INFO: `/v3/projects/c589204953414e7c823f39c050a26a88/users/88e3a87e53ae4535b31865d91195f2ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.504632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c589204953414e7c823f39c050a26a88, user_id=88e3a87e53ae4535b31865d91195f2ea, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.507336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.522121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1112/2238] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:57 2022] PUT /identity/v3/projects/c589204953414e7c823f39c050a26a88/users/88e3a87e53ae4535b31865d91195f2ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:57.526095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.538297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf3f6d-5250-4887-86b8-633cf80a4834 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.597506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc7ec1-1ee0-433d-be74-e348652f5bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.598313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc7ec1-1ee0-433d-be74-e348652f5bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.598603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc7ec1-1ee0-433d-be74-e348652f5bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.598887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc7ec1-1ee0-433d-be74-e348652f5bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.599833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc7ec1-1ee0-433d-be74-e348652f5bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.601685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc7ec1-1ee0-433d-be74-e348652f5bf6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.612541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1127/2239] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:44:57.616785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.628320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0788-c085-42d8-81ef-ae217bbee61c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.633137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.634074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.634510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.634919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] PATH_INFO: `/v3/projects/c589204953414e7c823f39c050a26a88/users/88e3a87e53ae4535b31865d91195f2ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.637332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c589204953414e7c823f39c050a26a88, user_id=88e3a87e53ae4535b31865d91195f2ea, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.640709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.652472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1113/2240] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:44:57 2022] PUT /identity/v3/projects/c589204953414e7c823f39c050a26a88/users/88e3a87e53ae4535b31865d91195f2ea/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:44:57.664302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.664603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.664890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.693009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] MFA Rules not processed for user `88e3a87e53ae4535b31865d91195f2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:57.694268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:57.764077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1128/2241] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:57 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:57.776317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:57.778318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:57.847055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4e380-b8ad-49db-8f92-74fd9d6b38e5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:57.852195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a031a7-eb4e-4729-a80a-bc9f2bc1f76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:57.853062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a031a7-eb4e-4729-a80a-bc9f2bc1f76b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:57.853434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a031a7-eb4e-4729-a80a-bc9f2bc1f76b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:57.853821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a031a7-eb4e-4729-a80a-bc9f2bc1f76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:57.866859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a031a7-eb4e-4729-a80a-bc9f2bc1f76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:57.868329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a031a7-eb4e-4729-a80a-bc9f2bc1f76b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:57.880496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1114/2242] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:44:59.472862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:59.492015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:44:59.538313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3631201-1fa5-4b77-9bb7-165f20d87e71 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:59.542733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:59.543620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:59.543902 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:59.544184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] PATH_INFO: `/v3/users/0ba5fb7d7b4e41f9a34a3b02119e42e4` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:59.545384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ba5fb7d7b4e41f9a34a3b02119e42e4)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:59.547334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:59.601275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ba5fb7d7b4e41f9a34a3b02119e42e4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:59.601956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ba5fb7d7b4e41f9a34a3b02119e42e4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:59.608748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ba5fb7d7b4e41f9a34a3b02119e42e4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:59.609365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ba5fb7d7b4e41f9a34a3b02119e42e4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:59.617924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ba5fb7d7b4e41f9a34a3b02119e42e4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:44:59.635539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1129/2243] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:44:59 2022] DELETE /identity/v3/users/0ba5fb7d7b4e41f9a34a3b02119e42e4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:44:59.640057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:59.640621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:59.640742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:59.659051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:44:59.660248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:59.723006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1115/2244] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:44:59.732587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:44:59.734035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:44:59.792429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac432ac-9d8a-4f68-bb40-f37984d7bf25 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:44:59.796058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:44:59.796729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:44:59.797015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:44:59.797295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:44:59.807054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:44:59.808491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:44:59.818884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1130/2245] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:00.092818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:00.094335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:00.107396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbcf21-4ead-4fa8-b738-4cca77900f56 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:00.112791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:00.113779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:00.114136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:00.114487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] PATH_INFO: `/v3/projects/20ec87de09114441bdc57f3cae2b8e3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:00.115738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20ec87de09114441bdc57f3cae2b8e3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:00.117653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:00.163542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:00.167414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ec87de09114441bdc57f3cae2b8e3d', 'request_id': 'req-b37521a9-4bdc-4453-ac22-b863e2fff703'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.176074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ec87de09114441bdc57f3cae2b8e3d', 'request_id': 'req-b37521a9-4bdc-4453-ac22-b863e2fff703'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.183081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ec87de09114441bdc57f3cae2b8e3d', 'request_id': 'req-b37521a9-4bdc-4453-ac22-b863e2fff703'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.195158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] The token cache is being invalidate because project 20ec87de09114441bdc57f3cae2b8e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:00.195449 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.196068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.197597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1116/2246] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:00 2022] DELETE /identity/v3/projects/20ec87de09114441bdc57f3cae2b8e3d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:00.201515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:00.211438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767f0b0-29c9-41e2-a20e-1cdb7a020b0a service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:00.251234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:00.273114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:00.274035 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:00.274321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:00.274598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] PATH_INFO: `/v3/users/88e3a87e53ae4535b31865d91195f2ea` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:00.275818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e3a87e53ae4535b31865d91195f2ea)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:00.277774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:00.334880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e3a87e53ae4535b31865d91195f2ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.335736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e3a87e53ae4535b31865d91195f2ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.347336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e3a87e53ae4535b31865d91195f2ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.348382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e3a87e53ae4535b31865d91195f2ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.356207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-34554247-f80c-47cf-b8ee-96fb7a1ebcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e3a87e53ae4535b31865d91195f2ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.379313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1131/2247] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:00 2022] DELETE /identity/v3/users/88e3a87e53ae4535b31865d91195f2ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:00.680257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:00.681900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:00.702230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37521a9-4bdc-4453-ac22-b863e2fff703 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:00.774530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:00.775404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:00.775717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:00.775999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] PATH_INFO: `/v3/projects/c589204953414e7c823f39c050a26a88` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:00.777186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c589204953414e7c823f39c050a26a88)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:00.779158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:00.827326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c589204953414e7c823f39c050a26a88', 'request_id': 'req-78e00ce4-5c92-4557-8bcc-6df8e0efab86'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.833789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c589204953414e7c823f39c050a26a88', 'request_id': 'req-78e00ce4-5c92-4557-8bcc-6df8e0efab86'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.840482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c589204953414e7c823f39c050a26a88', 'request_id': 'req-78e00ce4-5c92-4557-8bcc-6df8e0efab86'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.853972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] The token cache is being invalidate because project c589204953414e7c823f39c050a26a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:00.854202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.854851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-78e00ce4-5c92-4557-8bcc-6df8e0efab86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:00.856417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1117/2248] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:00 2022] DELETE /identity/v3/projects/c589204953414e7c823f39c050a26a88 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:00.861725 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:00.862438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:00.862734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:00.881579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:00.882842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:00.945869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1132/2249] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:00.954641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:00.954939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:00.955301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:00.974399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:00.975615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:00.983671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1118/2250] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:00.987952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:00.997723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b648f5-72ee-414a-b642-5c12b31cf8b2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.002811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.003459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.003747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.004028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.004766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:01.005077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.006771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.019382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1133/2251] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:45:01.023853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.033232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364bb60-16b8-459f-b9ce-ee560fd92e49 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.038018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.038674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.038968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.039249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.039985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.041900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.067903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1119/2252] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:01.072351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.086455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97795-8d75-4e86-b2ff-b6e49643a4cd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.130810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:01.145113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.145825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.146115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.146395 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.147089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.149011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.168206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1134/2253] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:01.172548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.182950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4310b2-a0be-4b66-9e96-b0715bb46faa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.187610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.188270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.188558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.188838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.190105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.191778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.202628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1120/2254] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:01.207589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.217874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27882eb8-0a81-4d31-9a63-85a98e125554 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.222730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25db4f-1f80-42fc-98d3-5b0b20fde69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.223393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25db4f-1f80-42fc-98d3-5b0b20fde69f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.223776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25db4f-1f80-42fc-98d3-5b0b20fde69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.224066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25db4f-1f80-42fc-98d3-5b0b20fde69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.224798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25db4f-1f80-42fc-98d3-5b0b20fde69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.226808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25db4f-1f80-42fc-98d3-5b0b20fde69f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.238573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1135/2255] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:01.242849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.253422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bd4be-f1d6-44f7-a26f-2a8ef2676a46 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.258103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.258986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.259275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.259561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] PATH_INFO: `/v3/projects/1780578927964854a9e7229e296d80c0/users/204f8bc8325549d7b8afacd270d0da74/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.274571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1780578927964854a9e7229e296d80c0, user_id=204f8bc8325549d7b8afacd270d0da74, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.277216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.286291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:01.292006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1121/2256] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:01 2022] PUT /identity/v3/projects/1780578927964854a9e7229e296d80c0/users/204f8bc8325549d7b8afacd270d0da74/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:01.303313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.303615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.303904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.330201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] MFA Rules not processed for user `204f8bc8325549d7b8afacd270d0da74`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:01.331438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:01.398458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1136/2257] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:01.402996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.468312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ae1e-034a-454b-a82f-e32b3c1232a0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.474232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.475322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.475322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.475666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.476687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.478577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.510814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1122/2258] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:01.512264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.523477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25216b-6f80-4e06-ae0b-3b144aa74a69 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.582285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.582975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.583264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.583548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.584291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.586227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.608861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1137/2259] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:01.613518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.624295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296558f3-6b0d-4a4f-a5ed-10bc53da936a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.629930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.630617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.630987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.631278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.632022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.633622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.644734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1123/2260] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:01.648662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.658947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74319d56-3a27-43cd-8f92-7423e73f19f2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.664013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ecdbb-7ce1-40dc-a1cf-033fa394ac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.664669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ecdbb-7ce1-40dc-a1cf-033fa394ac7f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.664953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ecdbb-7ce1-40dc-a1cf-033fa394ac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.665232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ecdbb-7ce1-40dc-a1cf-033fa394ac7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.665996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ecdbb-7ce1-40dc-a1cf-033fa394ac7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.667580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ecdbb-7ce1-40dc-a1cf-033fa394ac7f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.679085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1138/2261] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:01.683066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.693599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166893-74e6-4b94-81f4-9b0054123e8b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.698248 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.699133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.699452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.699757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] PATH_INFO: `/v3/projects/c2575ca9751b43fb87afd027a0234aa1/users/2b329bf6434847d8adcfa57fa7110c61/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.714654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2575ca9751b43fb87afd027a0234aa1, user_id=2b329bf6434847d8adcfa57fa7110c61, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.717288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.731035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1124/2262] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:01 2022] PUT /identity/v3/projects/c2575ca9751b43fb87afd027a0234aa1/users/2b329bf6434847d8adcfa57fa7110c61/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:01.741940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.742261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.742549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.768829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] MFA Rules not processed for user `2b329bf6434847d8adcfa57fa7110c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:01.770154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:01.791239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:01.840711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1139/2263] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:01.845221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.909644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54ac0c-ea32-478c-b640-4ad874045940 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:01.914360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:01.915040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:01.915332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:01.915618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:01.916606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:01.918273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:01.946075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1125/2264] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:01.949996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:01.960860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39472f87-fc63-44d8-a402-201e0cea5080 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:02.017079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:02.017806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.018138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.018445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.019131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:02.021038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:02.040587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1140/2265] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:01 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:02.044726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:02.055798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b68c-7dc2-4a22-8588-5425da7d8008 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:02.060404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:02.061065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.061359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.061651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.062410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:02.064053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:02.074613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1126/2266] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:02.078785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:02.089843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91952-93a4-495b-97b7-ef45a66dde3c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:02.094489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:02.095374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.095667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.096686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] PATH_INFO: `/v3/projects/ab21fae3e3be472b8305ec1d37ee8ed1/users/4c9267c3190643dc8856d1bffc7cc89c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.111026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab21fae3e3be472b8305ec1d37ee8ed1, user_id=4c9267c3190643dc8856d1bffc7cc89c, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:02.114337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:02.128161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1141/2267] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:02 2022] PUT /identity/v3/projects/ab21fae3e3be472b8305ec1d37ee8ed1/users/4c9267c3190643dc8856d1bffc7cc89c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:02.132066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:02.141972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a340d9-1193-4bac-b6d8-dcb5b98eeb2e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:02.161440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:02.199257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663760e9-0666-4972-824b-9be0471603a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:02.199926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663760e9-0666-4972-824b-9be0471603a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.200215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663760e9-0666-4972-824b-9be0471603a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.200502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663760e9-0666-4972-824b-9be0471603a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.201233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663760e9-0666-4972-824b-9be0471603a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:02.202862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663760e9-0666-4972-824b-9be0471603a9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:02.212453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1127/2268] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:02.216841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:02.227052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601078dc-bcc1-4f43-8db8-06e41b5ec58a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:02.231636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:02.232497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.232783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.233059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] PATH_INFO: `/v3/projects/ab21fae3e3be472b8305ec1d37ee8ed1/users/4c9267c3190643dc8856d1bffc7cc89c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.235204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab21fae3e3be472b8305ec1d37ee8ed1, user_id=4c9267c3190643dc8856d1bffc7cc89c, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:02.238702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:02.248784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1142/2269] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:02 2022] PUT /identity/v3/projects/ab21fae3e3be472b8305ec1d37ee8ed1/users/4c9267c3190643dc8856d1bffc7cc89c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:02.260753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.261059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.261354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.290972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] MFA Rules not processed for user `4c9267c3190643dc8856d1bffc7cc89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:02.292204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:02.362075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1128/2270] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:02.374363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:02.376094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:02.437515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89667644-fb0b-48f8-b0eb-371b1e5b6418 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:02.441611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:02.442276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:02.442614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:02.442957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:02.457944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:02.459581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:02.470208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1143/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:03.042825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:03.053499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c687-3f40-4f86-b766-8dc9cf08a415 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:03.058859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3ea1a-4c9e-4b23-81d0-585d887c5bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:03.059527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3ea1a-4c9e-4b23-81d0-585d887c5bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:03.059818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3ea1a-4c9e-4b23-81d0-585d887c5bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:03.060134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3ea1a-4c9e-4b23-81d0-585d887c5bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:03.128653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3ea1a-4c9e-4b23-81d0-585d887c5bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:03.131343 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3ea1a-4c9e-4b23-81d0-585d887c5bc4 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:03.142778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1129/2272] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:06.393681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:06.395424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:06.416208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:06.470720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2564-3179-4cc3-967c-ec7ccd81df9f service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:06.476137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:06.477048 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:06.477338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:06.477621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] PATH_INFO: `/v3/users/16698e1bead8473095b34a078018e625` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:06.478904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] RBAC: Authorizing `identity:delete_user(user_id=16698e1bead8473095b34a078018e625)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:06.480821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:06.527624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16698e1bead8473095b34a078018e625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:06.528229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16698e1bead8473095b34a078018e625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:06.535736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16698e1bead8473095b34a078018e625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:06.536391 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16698e1bead8473095b34a078018e625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:06.543767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16698e1bead8473095b34a078018e625'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:06.561922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1144/2273] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:06 2022] DELETE /identity/v3/users/16698e1bead8473095b34a078018e625 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:06.566841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:06.567304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:06.567537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:06.586472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:06.588196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:06.633258 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:06.654565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1130/2274] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:06.665029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:06.666493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:06.737051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa613a4-2a95-43a9-b0e5-e9e7678de52f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:06.741529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992da4-ec79-4838-b1c5-db4d3f19c1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:06.742556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992da4-ec79-4838-b1c5-db4d3f19c1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:06.742937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992da4-ec79-4838-b1c5-db4d3f19c1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:06.743310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992da4-ec79-4838-b1c5-db4d3f19c1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:06.753382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992da4-ec79-4838-b1c5-db4d3f19c1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:06.755152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992da4-ec79-4838-b1c5-db4d3f19c1d1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:06.767317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1145/2275] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:07.216320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.218393 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.231052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531d06e-b9b3-4d49-a30d-c1fcb824f0d8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.236063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.236950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.237382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.237689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] PATH_INFO: `/v3/projects/b5044c2755de4142a44628c1d8058537` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.239694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5044c2755de4142a44628c1d8058537)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.241617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.308001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5044c2755de4142a44628c1d8058537', 'request_id': 'req-76407ad3-7c30-4bd4-86f5-6a4178709ca2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.313795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5044c2755de4142a44628c1d8058537', 'request_id': 'req-76407ad3-7c30-4bd4-86f5-6a4178709ca2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.320303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5044c2755de4142a44628c1d8058537', 'request_id': 'req-76407ad3-7c30-4bd4-86f5-6a4178709ca2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.333601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] The token cache is being invalidate because project b5044c2755de4142a44628c1d8058537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:07.333947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.334684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76407ad3-7c30-4bd4-86f5-6a4178709ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:07.336372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1131/2276] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:07 2022] DELETE /identity/v3/projects/b5044c2755de4142a44628c1d8058537 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:07.341620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.341988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.342268 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.359828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:07.361011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.395370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:07.422659 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1146/2277] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:07.430334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.430667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.430964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.449142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:07.450386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.459104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1132/2278] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:07.463531 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.473772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30785513-3976-4ead-9b79-fa7c0bca8795 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.478479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.479140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.479427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.479713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.480475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:07.480788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.482403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.494233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1147/2279] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:07.498505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.508528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4379b-7148-4d2c-aa68-7fd6b5e9fdd0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.513739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.514410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.514698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.514979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.515687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.517621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.549392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1133/2280] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:07.550713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.560628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c29ef-04fb-4567-837a-cca29904792c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.619435 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.620116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.620405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.620686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.621380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.623321 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.647876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1148/2281] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:07.652240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.663115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae509-e207-4e44-ab62-bbe8838d1806 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.667762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.668421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.668708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.668991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.669752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.671345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.682173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1134/2282] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:07.686019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.696702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aab47-0cde-40ae-86d7-af9907b079ce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.701329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a2f6c-5db8-4b8e-9d4f-0b77bfa89094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.702025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2f6c-5db8-4b8e-9d4f-0b77bfa89094 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.702317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2f6c-5db8-4b8e-9d4f-0b77bfa89094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.702601 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2f6c-5db8-4b8e-9d4f-0b77bfa89094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.703338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2f6c-5db8-4b8e-9d4f-0b77bfa89094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.704930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2f6c-5db8-4b8e-9d4f-0b77bfa89094 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.715708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1149/2283] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:07.719725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.729895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d975f-61ee-4c92-8fb3-e7227ea486f8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.734503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.735380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.735670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.735955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] PATH_INFO: `/v3/projects/b9b31948ec124ad59e1ce1bf3c33c7d9/users/6f19f43e23a449999e73562567763f76/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.750641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b31948ec124ad59e1ce1bf3c33c7d9, user_id=6f19f43e23a449999e73562567763f76, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.753293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.766763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1135/2284] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:07 2022] PUT /identity/v3/projects/b9b31948ec124ad59e1ce1bf3c33c7d9/users/6f19f43e23a449999e73562567763f76/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:07.778479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.778807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.779089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.806657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] MFA Rules not processed for user `6f19f43e23a449999e73562567763f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:07.808204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.878107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1150/2285] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:07.890381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:07.892304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:07.949686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ada76-7deb-46f5-920e-383a4db0fe44 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:07.953326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:07.954006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:07.954295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:07.954575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:07.964464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:07.965886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:07.975240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1136/2286] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:08.502163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:08.512073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf54aa-720f-4b4f-9df2-a5a816497711 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:08.515814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:08.516538 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:08.516895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:08.517248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:08.519302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:08.555045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:08.582964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:08.584406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b038d4-e6a1-418c-b8d8-deeb48baf2b6 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:08.593729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1151/2287] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:10.864104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:10.866884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:10.875465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:11.016415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05693333-0d86-43b5-8ce6-0b766c834c75 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.022533 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.023428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.023719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.024007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] PATH_INFO: `/v3/users/204f8bc8325549d7b8afacd270d0da74` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.026488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204f8bc8325549d7b8afacd270d0da74)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.027548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.081402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204f8bc8325549d7b8afacd270d0da74'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.095489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204f8bc8325549d7b8afacd270d0da74'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.106536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204f8bc8325549d7b8afacd270d0da74'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.116969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204f8bc8325549d7b8afacd270d0da74'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.117703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204f8bc8325549d7b8afacd270d0da74'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.124755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1137/2288] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:10 2022] DELETE /identity/v3/users/204f8bc8325549d7b8afacd270d0da74 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:11.129234 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.129751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.129860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.166225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:11.167469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.235545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1152/2289] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:11.245907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.247416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.277886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:11.354317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9431-2109-4eb2-a940-82b4e16cb793 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.358106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66b251-4bd7-434f-8770-b6293fdb260a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.358820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66b251-4bd7-434f-8770-b6293fdb260a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.359111 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66b251-4bd7-434f-8770-b6293fdb260a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.359393 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66b251-4bd7-434f-8770-b6293fdb260a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.372954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66b251-4bd7-434f-8770-b6293fdb260a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.374489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66b251-4bd7-434f-8770-b6293fdb260a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.385005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1138/2290] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:11.575609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.577078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.591418 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd865a5-fd40-4998-abec-ff185cf53905 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.596741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.597802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.598088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.598367 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] PATH_INFO: `/v3/users/6f19f43e23a449999e73562567763f76` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.599647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f19f43e23a449999e73562567763f76)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.601718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.653824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f19f43e23a449999e73562567763f76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.654568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f19f43e23a449999e73562567763f76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.662752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f19f43e23a449999e73562567763f76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.663489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f19f43e23a449999e73562567763f76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.672647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f19f43e23a449999e73562567763f76'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.702978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1153/2291] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:11 2022] DELETE /identity/v3/users/6f19f43e23a449999e73562567763f76 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:11.707687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.707914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.708198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.728552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:11.729833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.792809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1139/2292] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:11.803615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.805033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.851950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:11.863809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.865743 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:11.870421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166528e-aa80-40a1-a766-369dcf632646 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.874655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.875284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.875573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.875850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.883309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530aa9e2-4a86-4632-a830-f66bf14cdb00 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:11.887539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.888100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.890282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:11.891159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:11.891441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:11.891722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] PATH_INFO: `/v3/projects/1780578927964854a9e7229e296d80c0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:11.893031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1780578927964854a9e7229e296d80c0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:11.894930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:11.900669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1154/2293] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:11.945136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1780578927964854a9e7229e296d80c0', 'request_id': 'req-76e2bca9-3b0c-41aa-b099-44331d334b57'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.951208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1780578927964854a9e7229e296d80c0', 'request_id': 'req-76e2bca9-3b0c-41aa-b099-44331d334b57'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.958990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1780578927964854a9e7229e296d80c0', 'request_id': 'req-76e2bca9-3b0c-41aa-b099-44331d334b57'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.973294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] The token cache is being invalidate because project 1780578927964854a9e7229e296d80c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:11.973842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.975423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:11.977071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1140/2294] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:11 2022] DELETE /identity/v3/projects/1780578927964854a9e7229e296d80c0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:11.981042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:11.991945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f490f-45dd-4173-bff1-3e87d81fc774 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.054527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.055430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.055727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.056004 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] PATH_INFO: `/v3/users/2b329bf6434847d8adcfa57fa7110c61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.057422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b329bf6434847d8adcfa57fa7110c61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.059451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.110383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b329bf6434847d8adcfa57fa7110c61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.111065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b329bf6434847d8adcfa57fa7110c61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.118839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b329bf6434847d8adcfa57fa7110c61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.119499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b329bf6434847d8adcfa57fa7110c61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.127495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c5afb4e5-a8b9-4f84-8176-0aae6987e198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b329bf6434847d8adcfa57fa7110c61'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.146183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1155/2295] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:11 2022] DELETE /identity/v3/users/2b329bf6434847d8adcfa57fa7110c61 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:12.173299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.174775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.189853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2bca9-3b0c-41aa-b099-44331d334b57 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.202732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:12.273805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.274547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.274908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.275279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] PATH_INFO: `/v3/projects/b9b31948ec124ad59e1ce1bf3c33c7d9` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.277801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b31948ec124ad59e1ce1bf3c33c7d9)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.279743 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.350031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b31948ec124ad59e1ce1bf3c33c7d9', 'request_id': 'req-b98b109a-530c-4cb5-912f-c6c8a0f49932'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.356308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b31948ec124ad59e1ce1bf3c33c7d9', 'request_id': 'req-b98b109a-530c-4cb5-912f-c6c8a0f49932'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.363044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b31948ec124ad59e1ce1bf3c33c7d9', 'request_id': 'req-b98b109a-530c-4cb5-912f-c6c8a0f49932'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.392914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] The token cache is being invalidate because project b9b31948ec124ad59e1ce1bf3c33c7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:12.394467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.395236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.397481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1141/2296] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:12 2022] DELETE /identity/v3/projects/b9b31948ec124ad59e1ce1bf3c33c7d9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:12.402788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb68fc-3145-44e1-a8c1-e5d3a779a9d0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.403091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb68fc-3145-44e1-a8c1-e5d3a779a9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.403376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb68fc-3145-44e1-a8c1-e5d3a779a9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.464444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-28eb68fc-3145-44e1-a8c1-e5d3a779a9d0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:12.465632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28eb68fc-3145-44e1-a8c1-e5d3a779a9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.499403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.501161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.603219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:12.629761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b109a-530c-4cb5-912f-c6c8a0f49932 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.636451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1156/2297] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:12.638558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.639667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.640024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.640373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] PATH_INFO: `/v3/projects/c2575ca9751b43fb87afd027a0234aa1` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.642007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2575ca9751b43fb87afd027a0234aa1)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.644752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.645153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.646476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.648752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.666634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:12.667943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.677701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1157/2298] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:12.680449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.703934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69be31-4398-462b-b0f5-06356d4dfcb9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.716160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.716868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.717146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.717425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.718204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:12.718520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.720574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.753257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:12.755524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2575ca9751b43fb87afd027a0234aa1', 'request_id': 'req-56da0752-6422-4445-bc8c-675b5568539d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.757312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1158/2299] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:12.762082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.768984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2575ca9751b43fb87afd027a0234aa1', 'request_id': 'req-56da0752-6422-4445-bc8c-675b5568539d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.777566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2575ca9751b43fb87afd027a0234aa1', 'request_id': 'req-56da0752-6422-4445-bc8c-675b5568539d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.787641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475c17d-fb76-44d5-97c6-09ea84467597 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.792789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] The token cache is being invalidate because project c2575ca9751b43fb87afd027a0234aa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:12.793088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.793787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.796140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1142/2300] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:12 2022] DELETE /identity/v3/projects/c2575ca9751b43fb87afd027a0234aa1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:12.799879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.801280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.853089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.853996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.854449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.854772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.855606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.857655 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.871665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da0752-6422-4445-bc8c-675b5568539d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.878561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.880952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.881320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.881611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] PATH_INFO: `/v3/users/4c9267c3190643dc8856d1bffc7cc89c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.883184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9267c3190643dc8856d1bffc7cc89c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.885268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.889424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1159/2301] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:12 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:12.893237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:12.895140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:12.944243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9267c3190643dc8856d1bffc7cc89c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.956813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9267c3190643dc8856d1bffc7cc89c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.964922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9267c3190643dc8856d1bffc7cc89c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.967681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c728fe-a910-40b5-a882-286f4a707137 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:12.973621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:12.974217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9267c3190643dc8856d1bffc7cc89c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.974496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:12.974823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:12.975154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:12.975442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9267c3190643dc8856d1bffc7cc89c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:12.976144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:12.978145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:12.982070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1143/2302] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:12 2022] DELETE /identity/v3/users/4c9267c3190643dc8856d1bffc7cc89c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:12.999384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1160/2303] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:12 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:13.003358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.004767 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.067504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:13.075022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56839c07-8fb9-4bd5-87fa-ef79fc2bc3c7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.081508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.082253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.082538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.082966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.084409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.086003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.097155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1144/2304] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:13.101165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.111275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c15f6-69f9-4ad9-8e22-a298bd572459 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.116376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db19f4-df60-4e91-8c8a-7c61a47bdd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.117070 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db19f4-df60-4e91-8c8a-7c61a47bdd95 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.117340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db19f4-df60-4e91-8c8a-7c61a47bdd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.117624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db19f4-df60-4e91-8c8a-7c61a47bdd95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.118396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db19f4-df60-4e91-8c8a-7c61a47bdd95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.120170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db19f4-df60-4e91-8c8a-7c61a47bdd95 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.131578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1161/2305] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:13.135716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.146784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97618dd3-298b-4277-b131-5f8960111c6d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.151836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.152394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.152679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.152957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] PATH_INFO: `/v3/projects/a57bab77e60b4efc9df43e95f1ead38b/users/7d6eaf2c133743a08314ca0d9dcc59a8/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.168287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a57bab77e60b4efc9df43e95f1ead38b, user_id=7d6eaf2c133743a08314ca0d9dcc59a8, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.170973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.184537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1145/2306] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:13 2022] PUT /identity/v3/projects/a57bab77e60b4efc9df43e95f1ead38b/users/7d6eaf2c133743a08314ca0d9dcc59a8/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:13.196170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.196687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.196777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.225000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] MFA Rules not processed for user `7d6eaf2c133743a08314ca0d9dcc59a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:13.226120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.245855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:13.274711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.276708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.305586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1162/2307] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:13.313255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.315146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.360607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478127-68cb-429e-a1a4-c271811ddaed admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.366197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.367073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.367367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.367647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] PATH_INFO: `/v3/projects/ab21fae3e3be472b8305ec1d37ee8ed1` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.368990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab21fae3e3be472b8305ec1d37ee8ed1)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.370991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.376466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6b82-b781-4554-b93c-0f10ce9d420d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.380893 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa12e1-4c5b-4762-9411-a80befcf8264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.381457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa12e1-4c5b-4762-9411-a80befcf8264 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.381782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa12e1-4c5b-4762-9411-a80befcf8264 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.382064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa12e1-4c5b-4762-9411-a80befcf8264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.391541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa12e1-4c5b-4762-9411-a80befcf8264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.392990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa12e1-4c5b-4762-9411-a80befcf8264 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.402795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1163/2308] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:13.421653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab21fae3e3be472b8305ec1d37ee8ed1', 'request_id': 'req-faa43da3-c26c-4b7a-b63a-b63108390089'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.428215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab21fae3e3be472b8305ec1d37ee8ed1', 'request_id': 'req-faa43da3-c26c-4b7a-b63a-b63108390089'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.434624 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab21fae3e3be472b8305ec1d37ee8ed1', 'request_id': 'req-faa43da3-c26c-4b7a-b63a-b63108390089'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.448606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] The token cache is being invalidate because project ab21fae3e3be472b8305ec1d37ee8ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:13.448903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.449588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-faa43da3-c26c-4b7a-b63a-b63108390089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:13.451386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1146/2309] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:13 2022] DELETE /identity/v3/projects/ab21fae3e3be472b8305ec1d37ee8ed1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:13.457121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.457422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.457730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.478493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:13.479722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.540525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1164/2310] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:13.549040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.549525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.550093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.574455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:13.575682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:13.584267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1147/2311] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:13.588879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.599568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d12a4f-d34d-44e5-9bcd-d88789fed2f8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.604762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.605434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.605758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.606043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.607403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:13.607727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.609908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.626957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1165/2312] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:45:13.631510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.642318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13215e32-4a10-4851-8a64-8f162ab5056e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.647512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.655628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.655988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.656278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.657132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.658950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.691779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1148/2313] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:13 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:13.696301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.707443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd736a4-2728-4b84-bc3e-e1cf2889d82f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.778567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:13.790628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.790845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.791113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.791403 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.792108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.794290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.826552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1166/2314] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:13 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:13.830804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.861633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a3aab-2548-475b-9643-e929cb026117 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.869827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.870653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.871013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.871400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.872308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.874084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.888826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1149/2315] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:13.890085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.901492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa8720-9036-452e-ada8-490f93371f94 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.907915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf2c95-27ed-4f56-865b-8a57fba79836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.908605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2c95-27ed-4f56-865b-8a57fba79836 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.909034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2c95-27ed-4f56-865b-8a57fba79836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.909325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2c95-27ed-4f56-865b-8a57fba79836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.910106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf2c95-27ed-4f56-865b-8a57fba79836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.912591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf2c95-27ed-4f56-865b-8a57fba79836 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.930185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1167/2316] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:13.934734 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:13.946367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe1c2e-f18d-467c-b1b0-115407231190 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:13.952637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:13.953350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:13.953754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:13.954002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] PATH_INFO: `/v3/projects/40cf20b7c284419fbf7d5b769fcc6244/users/0f9ac35e1181462f9068fd2ec29ee405/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:13.986714 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cf20b7c284419fbf7d5b769fcc6244, user_id=0f9ac35e1181462f9068fd2ec29ee405, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:13.989424 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:13.993031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:14.013763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1150/2317] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:13 2022] PUT /identity/v3/projects/40cf20b7c284419fbf7d5b769fcc6244/users/0f9ac35e1181462f9068fd2ec29ee405/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:14.019754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.020055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.020397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.050866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] MFA Rules not processed for user `0f9ac35e1181462f9068fd2ec29ee405`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:14.052151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:14.125905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1168/2318] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:14.130558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.205014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98dd16-f4c3-41cf-aede-532c2b4e177d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.210376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.211042 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.211337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.211622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.212335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.214534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.242822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1151/2319] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:14.246743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.257910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34127aec-c6e5-4d04-b7ed-b76a62b76a99 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.317685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.318376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.318669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.318953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.319667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.321844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.343230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1169/2320] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:14.347062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.359468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a07b-dbca-4fa7-9b46-a170a305c87b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.364104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.365350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.365700 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.365990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.366724 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.368316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.379626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1152/2321] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:14.384586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.394822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1620d-d38c-4273-a5d8-e08f7ab36df4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.400433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.401385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.401695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.402192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] PATH_INFO: `/v3/projects/7005225e30eb4cfe9da7c5264a132c2e/users/154d5b4eaeb548b9b0699e341c9b2d42/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.418801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7005225e30eb4cfe9da7c5264a132c2e, user_id=154d5b4eaeb548b9b0699e341c9b2d42, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.421424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.435784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1170/2322] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:14 2022] PUT /identity/v3/projects/7005225e30eb4cfe9da7c5264a132c2e/users/154d5b4eaeb548b9b0699e341c9b2d42/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:14.439895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.454388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8966206-82ec-464d-b901-88e3e759f2c8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.517195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e6996-1e8c-4024-85d3-3fdeeb9dbb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.517900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e6996-1e8c-4024-85d3-3fdeeb9dbb21 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.518195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e6996-1e8c-4024-85d3-3fdeeb9dbb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.518476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e6996-1e8c-4024-85d3-3fdeeb9dbb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.519210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e6996-1e8c-4024-85d3-3fdeeb9dbb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.520872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e6996-1e8c-4024-85d3-3fdeeb9dbb21 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.531266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1153/2323] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:14.535218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.546645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd3bf2-8973-4b37-b4f5-8192edd5631a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.551981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.553847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.554140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.554422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] PATH_INFO: `/v3/projects/7005225e30eb4cfe9da7c5264a132c2e/users/154d5b4eaeb548b9b0699e341c9b2d42/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.556688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7005225e30eb4cfe9da7c5264a132c2e, user_id=154d5b4eaeb548b9b0699e341c9b2d42, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.559409 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.571420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1171/2324] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:14 2022] PUT /identity/v3/projects/7005225e30eb4cfe9da7c5264a132c2e/users/154d5b4eaeb548b9b0699e341c9b2d42/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:14.582206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f11abf-5bb8-4eee-b3c1-a34ca447873a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.582506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f11abf-5bb8-4eee-b3c1-a34ca447873a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.582793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f11abf-5bb8-4eee-b3c1-a34ca447873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.611794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b1f11abf-5bb8-4eee-b3c1-a34ca447873a None None] MFA Rules not processed for user `154d5b4eaeb548b9b0699e341c9b2d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:14.613027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b1f11abf-5bb8-4eee-b3c1-a34ca447873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:14.666323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:14.688648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1154/2325] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:14.699482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.701171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:14.715953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:14.763282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efefe2a-377a-4bbc-818e-06e45fe91214 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.767306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.768197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.768492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.768785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.779579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:14.781015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:14.791987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1172/2326] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:14.879344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.879344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.879820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.900876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] MFA Rules not processed for user `7d6eaf2c133743a08314ca0d9dcc59a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:14.902222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:14.954715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:14.966744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994438a-879f-487f-9576-604fb7a91cf2 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:14.972290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30683ec-f845-465f-824a-451e127260d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:14.973257 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30683ec-f845-465f-824a-451e127260d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:14.974738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30683ec-f845-465f-824a-451e127260d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:14.975519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30683ec-f845-465f-824a-451e127260d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:14.984125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1155/2327] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:15.043455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30683ec-f845-465f-824a-451e127260d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:15.044934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30683ec-f845-465f-824a-451e127260d3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:15.055311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1173/2328] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:20.843515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:20.924995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca7fc-1926-418a-923e-ba4299989e96 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:20.930997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:20.931997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:20.932334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:20.932621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] PATH_INFO: `/v3/users/0f9ac35e1181462f9068fd2ec29ee405` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:20.935294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9ac35e1181462f9068fd2ec29ee405)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:20.936107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:21.003458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9ac35e1181462f9068fd2ec29ee405'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.023300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9ac35e1181462f9068fd2ec29ee405'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.031283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9ac35e1181462f9068fd2ec29ee405'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.039940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9ac35e1181462f9068fd2ec29ee405'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.040696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9ac35e1181462f9068fd2ec29ee405'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.047387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1156/2329] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:20 2022] DELETE /identity/v3/users/0f9ac35e1181462f9068fd2ec29ee405 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:21.051693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:21.052192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:21.052299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:21.070777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:21.071908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:21.133933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1174/2330] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:21.143490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:21.144917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:21.159688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:21.202704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbae7fa-9450-4497-8c85-041143bf44a3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:21.206374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:21.207023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:21.207317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:21.207602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:21.217518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:21.219003 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:21.228911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1157/2331] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:21.596374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:21.598635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:21.613152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73ee5-c9d3-409d-afca-9ec4391dba4f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:21.619561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:21.620892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:21.620892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:21.621250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] PATH_INFO: `/v3/projects/40cf20b7c284419fbf7d5b769fcc6244` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:21.622377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cf20b7c284419fbf7d5b769fcc6244)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:21.624904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:21.711197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:21.712811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cf20b7c284419fbf7d5b769fcc6244', 'request_id': 'req-231591b1-4aea-495a-806c-b63edc4ee73f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.723307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cf20b7c284419fbf7d5b769fcc6244', 'request_id': 'req-231591b1-4aea-495a-806c-b63edc4ee73f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.731983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cf20b7c284419fbf7d5b769fcc6244', 'request_id': 'req-231591b1-4aea-495a-806c-b63edc4ee73f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.746059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] The token cache is being invalidate because project 40cf20b7c284419fbf7d5b769fcc6244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:21.746465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.747201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.748762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1175/2332] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:21 2022] DELETE /identity/v3/projects/40cf20b7c284419fbf7d5b769fcc6244 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:21.753081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:21.763071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e33d0-147f-46f1-be35-bc1780edc717 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:21.835798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:21.836610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:21.836894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:21.837170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] PATH_INFO: `/v3/users/154d5b4eaeb548b9b0699e341c9b2d42` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:21.838753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=154d5b4eaeb548b9b0699e341c9b2d42)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:21.840665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:21.892851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154d5b4eaeb548b9b0699e341c9b2d42'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.908840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154d5b4eaeb548b9b0699e341c9b2d42'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.917500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154d5b4eaeb548b9b0699e341c9b2d42'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.931106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154d5b4eaeb548b9b0699e341c9b2d42'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.932118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a950deb5-c0fc-45bf-84e7-78d076921c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154d5b4eaeb548b9b0699e341c9b2d42'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:21.940703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1158/2333] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:21 2022] DELETE /identity/v3/users/154d5b4eaeb548b9b0699e341c9b2d42 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:22.426407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.429920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.451802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231591b1-4aea-495a-806c-b63edc4ee73f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:22.526471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:22.527354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.527644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.527930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] PATH_INFO: `/v3/projects/7005225e30eb4cfe9da7c5264a132c2e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.529316 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7005225e30eb4cfe9da7c5264a132c2e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:22.531343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:22.580622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7005225e30eb4cfe9da7c5264a132c2e', 'request_id': 'req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.589045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7005225e30eb4cfe9da7c5264a132c2e', 'request_id': 'req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.594572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7005225e30eb4cfe9da7c5264a132c2e', 'request_id': 'req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.607294 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] The token cache is being invalidate because project 7005225e30eb4cfe9da7c5264a132c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:22.607516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.608226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0fc7bc88-0d7c-479c-b7a4-bf1df9847c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:22.610669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1176/2334] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:22 2022] DELETE /identity/v3/projects/7005225e30eb4cfe9da7c5264a132c2e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:22.616828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.617199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.617545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.656740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:22.658414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.690277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:22.754148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1159/2335] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:22.762107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.762406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.762698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.787245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:22.788801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:22.797104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1177/2336] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:22.801745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.813116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365b00-0db0-4054-bcae-f36da4786438 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:22.818140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:22.818970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.819441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.819729 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.820558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:22.820911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:22.822808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:22.838396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1160/2337] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:22.839794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.850576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77147d-3f44-409f-8532-11f894f50462 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:22.855287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:22.855970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.856385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.856674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.857431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:22.859309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:22.889793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1178/2338] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:22 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:22.891180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:22.901632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3a5db-a1d2-4246-b839-52ef8d1a6e06 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:22.958408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:22.959597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:22.959887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:22.960170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:22.960869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:22.962821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:22.983671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1161/2339] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:22 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:22.987967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.000410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be6bec-ef24-46b8-87bf-a7f1b8ff46e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.005240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.007153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.007439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.007720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.008448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.010071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.022394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1179/2340] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:23.026375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.036544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6069f70-e409-412c-9153-be3afae18217 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.042601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d0484-e72a-4518-bf37-32cdfd9c19c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.043314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d0484-e72a-4518-bf37-32cdfd9c19c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.043616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d0484-e72a-4518-bf37-32cdfd9c19c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.043875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d0484-e72a-4518-bf37-32cdfd9c19c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.044603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d0484-e72a-4518-bf37-32cdfd9c19c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.046224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d0484-e72a-4518-bf37-32cdfd9c19c1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.057471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1162/2341] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:23.061460 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.072053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360a539-3cdd-4ff4-8a48-b65116dc6547 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.076736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.077648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.077959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.078242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] PATH_INFO: `/v3/projects/e065980f79fa48c588acf46b023acc1f/users/b4db7589b0524e368cd71e055e2072eb/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.093789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e065980f79fa48c588acf46b023acc1f, user_id=b4db7589b0524e368cd71e055e2072eb, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.096837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.110310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1180/2342] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:23 2022] PUT /identity/v3/projects/e065980f79fa48c588acf46b023acc1f/users/b4db7589b0524e368cd71e055e2072eb/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:23.122203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.123027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.123516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.155009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] MFA Rules not processed for user `b4db7589b0524e368cd71e055e2072eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:23.156242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:23.227540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1163/2343] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:23.240144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:23.242023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:23.247855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:23.309003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6672c2-9cec-422b-805e-f8b30ddcd150 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:23.312702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c89e9-60f3-419f-8c61-8c535005aea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:23.313361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c89e9-60f3-419f-8c61-8c535005aea0 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:23.313648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c89e9-60f3-419f-8c61-8c535005aea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:23.313955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c89e9-60f3-419f-8c61-8c535005aea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:23.323375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c89e9-60f3-419f-8c61-8c535005aea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:23.324836 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c89e9-60f3-419f-8c61-8c535005aea0 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:23.334777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1181/2344] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:24.971976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:24.973870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:25.002483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:25.044197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506c328-8074-4daa-9b51-89b14b3587e8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.048717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.049597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.049919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.050203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] PATH_INFO: `/v3/users/0db68ec309c44989a2b9d373da3294c5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.051403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db68ec309c44989a2b9d373da3294c5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.053359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.100076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db68ec309c44989a2b9d373da3294c5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.111959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db68ec309c44989a2b9d373da3294c5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.119400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db68ec309c44989a2b9d373da3294c5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.127330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db68ec309c44989a2b9d373da3294c5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.128017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db68ec309c44989a2b9d373da3294c5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.134571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1164/2345] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:24 2022] DELETE /identity/v3/users/0db68ec309c44989a2b9d373da3294c5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:25.139219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.139695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.139815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.157788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:25.159006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:25.219115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1182/2346] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:25.228903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.230405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:25.288305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce627c1-2b6d-401b-80c6-3268ead471f0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.292451 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.293156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.293556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.293943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.305888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.307306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.316659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1165/2347] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:25.630352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:25.632359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:25.645094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59805163-177f-497d-8b44-c5152a4d64bb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:25.651555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:25.652472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:25.652763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:25.653043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] PATH_INFO: `/v3/projects/95ad99f5649a4af8965d258b748ad428` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:25.654284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ad99f5649a4af8965d258b748ad428)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:25.656245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:25.707289 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ad99f5649a4af8965d258b748ad428', 'request_id': 'req-2aef9136-906e-449b-a412-f55d8af694ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.715772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ad99f5649a4af8965d258b748ad428', 'request_id': 'req-2aef9136-906e-449b-a412-f55d8af694ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.722044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ad99f5649a4af8965d258b748ad428', 'request_id': 'req-2aef9136-906e-449b-a412-f55d8af694ce'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.734750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] The token cache is being invalidate because project 95ad99f5649a4af8965d258b748ad428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:25.735113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.735771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:25.737333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1183/2348] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:25 2022] DELETE /identity/v3/projects/95ad99f5649a4af8965d258b748ad428 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:35.312573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:35.731618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:36.034984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:36.043694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0d270-de46-4ae0-8f89-98add933ac89 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:36.097187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:36.097870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:36.098160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:36.098442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:36.100937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:36.163992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:36.165430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b85ad-6888-4d83-8c28-aff4cc236711 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:36.175093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1166/2349] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:46.170566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=50 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:47.528005 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:47.529494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:47.592763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef9136-906e-449b-a412-f55d8af694ce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:47.597298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:47.598269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:47.598554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:47.598835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] PATH_INFO: `/v3/users/b4db7589b0524e368cd71e055e2072eb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:47.600015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4db7589b0524e368cd71e055e2072eb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:47.601966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:47.653609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4db7589b0524e368cd71e055e2072eb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:47.654291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4db7589b0524e368cd71e055e2072eb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:47.661440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4db7589b0524e368cd71e055e2072eb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:47.662046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4db7589b0524e368cd71e055e2072eb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:47.670736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4db7589b0524e368cd71e055e2072eb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:47.692470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1184/2350] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:47 2022] DELETE /identity/v3/users/b4db7589b0524e368cd71e055e2072eb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:47.694521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:47.694955 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:47.695092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:47.713541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:47.715057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:47.783212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1167/2351] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:47.793881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:47.795282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:47.851280 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855946f-5d1f-4779-8c94-e1ef525a4cf6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:47.855656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f86665-55ed-4367-931f-9e7441a70c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:47.856305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f86665-55ed-4367-931f-9e7441a70c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:47.856598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f86665-55ed-4367-931f-9e7441a70c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:47.856883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f86665-55ed-4367-931f-9e7441a70c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:47.866187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f86665-55ed-4367-931f-9e7441a70c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:47.867634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f86665-55ed-4367-931f-9e7441a70c27 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:47.877015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1185/2352] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:48.155289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.156772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:48.168964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca6ee8-b4a3-4775-8045-dbe9cc47e4da None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.174833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.174833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.175752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.176114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] PATH_INFO: `/v3/projects/e065980f79fa48c588acf46b023acc1f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.177336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e065980f79fa48c588acf46b023acc1f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.179301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.226531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e065980f79fa48c588acf46b023acc1f', 'request_id': 'req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:48.232184 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e065980f79fa48c588acf46b023acc1f', 'request_id': 'req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:48.238288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e065980f79fa48c588acf46b023acc1f', 'request_id': 'req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:48.250189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] The token cache is being invalidate because project e065980f79fa48c588acf46b023acc1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:48.250531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:48.251201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9b4c2551-aabe-4ea0-a751-564aa1b73a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:48.252725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1168/2353] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:48 2022] DELETE /identity/v3/projects/e065980f79fa48c588acf46b023acc1f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:48.257757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.258059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.258346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.277203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:48.278447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:48.295346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:48.346380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1186/2354] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:48.354506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.354808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.355097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.374321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:48.375545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:48.385578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1169/2355] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:48.392710 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.400920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fce94-40d8-4069-b540-d3590fbd8e20 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.405823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.406673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.406966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.407895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.410049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:48.410049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.411765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.425528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1187/2356] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:48.430141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.440754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25142a-7ea8-4c68-a25c-9fefa8cf82b7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.448134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.451004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.451335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.451631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.452470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.454612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.492144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1170/2357] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:48 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:48.497132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.507654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b128e-ae6f-4536-b9e5-fe4d01d405e8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.568860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.569544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.569867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.570151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.570853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.573582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.593483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1188/2358] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:48 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:48.597548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.607949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c295d-6235-421c-8aa6-dd1ea0bdae72 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.612544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.613190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.613505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.613825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.614551 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.616144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.627660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1171/2359] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:48.631663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.642842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8743b-5f0f-4f74-87d9-b3a1ca793918 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.647489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba4a19-ca5f-414d-b649-9bc86afe1029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.648157 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba4a19-ca5f-414d-b649-9bc86afe1029 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.648450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba4a19-ca5f-414d-b649-9bc86afe1029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.648762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba4a19-ca5f-414d-b649-9bc86afe1029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.649833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba4a19-ca5f-414d-b649-9bc86afe1029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.651625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba4a19-ca5f-414d-b649-9bc86afe1029 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.661826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1189/2360] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:48.666646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.677030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfac9af-6a56-420f-ac78-7b24181a7734 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.682408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.683299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.683597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.683880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] PATH_INFO: `/v3/projects/a6df891689d24cd483c6b37aa7810083/users/7371ef69d753407fbb83e0cdf5df6e0e/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.699906 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6df891689d24cd483c6b37aa7810083, user_id=7371ef69d753407fbb83e0cdf5df6e0e, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.702704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.717004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1172/2361] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:48 2022] PUT /identity/v3/projects/a6df891689d24cd483c6b37aa7810083/users/7371ef69d753407fbb83e0cdf5df6e0e/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:48.728488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.728812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.729099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.756149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] MFA Rules not processed for user `7371ef69d753407fbb83e0cdf5df6e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:48.757372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:48.829339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1190/2362] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:48.840461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:48.842289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:48.869470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:48.928670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3b0d7-d9d3-4dd5-a5d9-4297c4640158 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:48.933264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024512f-0d8a-4090-975a-1b8144ca79d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:48.933956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024512f-0d8a-4090-975a-1b8144ca79d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:48.934249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024512f-0d8a-4090-975a-1b8144ca79d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:48.934529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024512f-0d8a-4090-975a-1b8144ca79d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:48.943307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024512f-0d8a-4090-975a-1b8144ca79d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:48.944909 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024512f-0d8a-4090-975a-1b8144ca79d9 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:48.955097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1173/2363] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:51.980421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:52.007571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:52.061596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8e6b6-6555-4cd8-af69-3fbc42a55a1d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:52.067700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:52.068750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.069164 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.069993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] PATH_INFO: `/v3/users/7371ef69d753407fbb83e0cdf5df6e0e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.074991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7371ef69d753407fbb83e0cdf5df6e0e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:52.077118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:52.150025 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7371ef69d753407fbb83e0cdf5df6e0e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.150715 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7371ef69d753407fbb83e0cdf5df6e0e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.159283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7371ef69d753407fbb83e0cdf5df6e0e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.160152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7371ef69d753407fbb83e0cdf5df6e0e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.173551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7371ef69d753407fbb83e0cdf5df6e0e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.203079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1191/2364] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:51 2022] DELETE /identity/v3/users/7371ef69d753407fbb83e0cdf5df6e0e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:52.204253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1a63c-46b8-4a11-987a-c29396ab384b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.204743 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1a63c-46b8-4a11-987a-c29396ab384b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.204842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1a63c-46b8-4a11-987a-c29396ab384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.227957 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0fd1a63c-46b8-4a11-987a-c29396ab384b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:52.228658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0fd1a63c-46b8-4a11-987a-c29396ab384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.314927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1174/2365] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:52.324065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:52.325508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.364469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.365043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.365331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.392727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa78f7-8d10-4129-9f63-3926b34a02b3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:52.395778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:52.396411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:52.397073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.397204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.397364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.397652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.408505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:52.409890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:52.420293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1192/2366] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:52.464720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1175/2367] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:52.509972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:52.511577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.556600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:52.587929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3769e2e-a5d2-4a9f-9002-c3c6f5ded624 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:52.593445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f6d6f-983a-4d6f-9674-66771b588315 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:52.594142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f6d6f-983a-4d6f-9674-66771b588315 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.594436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f6d6f-983a-4d6f-9674-66771b588315 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.594719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f6d6f-983a-4d6f-9674-66771b588315 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.603156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f6d6f-983a-4d6f-9674-66771b588315 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:52.605079 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f6d6f-983a-4d6f-9674-66771b588315 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:52.613253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1193/2368] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:45:52.724369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:52.726206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.749807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01127fe6-bcc1-4166-82b5-349a0ef6493d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:52.755247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:52.756499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.757168 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.757803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] PATH_INFO: `/v3/projects/a6df891689d24cd483c6b37aa7810083` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.761101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6df891689d24cd483c6b37aa7810083)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:52.763779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:52.813746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:52.831464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6df891689d24cd483c6b37aa7810083', 'request_id': 'req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.838154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6df891689d24cd483c6b37aa7810083', 'request_id': 'req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.845612 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6df891689d24cd483c6b37aa7810083', 'request_id': 'req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.860579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] The token cache is being invalidate because project a6df891689d24cd483c6b37aa7810083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:52.860878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.861509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11ba837f-8107-4c7d-95e5-f436dcbf2ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:52.863428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1176/2369] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:52 2022] DELETE /identity/v3/projects/a6df891689d24cd483c6b37aa7810083 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:52.868283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.868589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.868909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.887318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:52.888535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.952235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1194/2370] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:52.960372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:52.960617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:52.960905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:52.979924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:52.981131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:52.989270 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1177/2371] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:52.994196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.004770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52e7a5-327e-4b1b-91fb-412237e3025d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.011102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.011805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.012100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.012378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.013118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:53.013434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.015065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.028701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1195/2372] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:53.033064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.043015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d466fa-95c7-41a6-921b-66e00c2804d7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.048642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.049312 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.049606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.049922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.050848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.052783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.084627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1178/2373] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:53 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:53.085914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.096306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28e809-6c4e-4bf9-94bd-209442cd5f7b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.161505 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dfb07-44a1-4d17-b7e1-e48d12462e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.162735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dfb07-44a1-4d17-b7e1-e48d12462e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.163023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dfb07-44a1-4d17-b7e1-e48d12462e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.163302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dfb07-44a1-4d17-b7e1-e48d12462e92 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.164012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dfb07-44a1-4d17-b7e1-e48d12462e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.169357 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dfb07-44a1-4d17-b7e1-e48d12462e92 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.189707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1196/2374] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:53 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:53.193680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.202021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.202322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.202632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.205894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed946-501d-442f-b453-da5a5916ce7c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.211199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.212350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.212776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.213175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.214069 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.216196 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.222443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:53.223817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:53.227950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1179/2375] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:53.231703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.242467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564eab2-3641-4c23-9938-c4739e198e77 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.249365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.254508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.254804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.255095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.256115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.257771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.271011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1180/2376] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:45:53.274948 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.278634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:53.287057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ebcb6-da4c-4650-a52a-a47fae052e29 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.293263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd716-5056-46fb-bf5a-f1635fce876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.294609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd716-5056-46fb-bf5a-f1635fce876d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.294609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd716-5056-46fb-bf5a-f1635fce876d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.294858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd716-5056-46fb-bf5a-f1635fce876d admin admin] PATH_INFO: `/v3/projects/2350f7b62a7d4d8199faf8b18b2ca8dc/users/67d64cad336c44d28c9cf31fc9896854/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.300237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1197/2377] 10.222.0.57 () {58 vars in 967 bytes} [Wed Jun 15 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:53.313485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd716-5056-46fb-bf5a-f1635fce876d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2350f7b62a7d4d8199faf8b18b2ca8dc, user_id=67d64cad336c44d28c9cf31fc9896854, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.316173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd716-5056-46fb-bf5a-f1635fce876d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.318115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.319574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:53.332805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1181/2378] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:53 2022] PUT /identity/v3/projects/2350f7b62a7d4d8199faf8b18b2ca8dc/users/67d64cad336c44d28c9cf31fc9896854/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:53.338278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.338573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.338854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.358431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:53.360062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:53.414441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e4856-fdf6-4f30-ae4f-532105df4b60 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.447494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1182/2379] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:53.457543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.459037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:53.484277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.484960 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.485256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.485544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.513740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:53.528227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c330da-72c5-41b3-b55a-ee074d9af3b2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.532074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878ebf4-34d0-4494-9fb9-cb2a8b80370f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.532735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878ebf4-34d0-4494-9fb9-cb2a8b80370f service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.533028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878ebf4-34d0-4494-9fb9-cb2a8b80370f service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.533315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878ebf4-34d0-4494-9fb9-cb2a8b80370f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.543479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878ebf4-34d0-4494-9fb9-cb2a8b80370f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.544912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878ebf4-34d0-4494-9fb9-cb2a8b80370f service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.555378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.556105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1183/2380] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:45:53.557789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.567068 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1198/2381] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:45:53.810166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbed4c-fe89-4ac8-8ad9-e7a732d7a01a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.810569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbed4c-fe89-4ac8-8ad9-e7a732d7a01a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.810985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbed4c-fe89-4ac8-8ad9-e7a732d7a01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.831372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-34cbed4c-fe89-4ac8-8ad9-e7a732d7a01a None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:53.832701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-34cbed4c-fe89-4ac8-8ad9-e7a732d7a01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:53.842378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1184/2382] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:45:53.855769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:53.868408 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5817b43-fe85-4e69-9d71-446a193b83aa service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:53.874082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:53.874845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:53.875239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:53.875656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:53.884900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:53.886634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:53.897182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1199/2383] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:45:54.377891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:54.378471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:54.378707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:54.417378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:54.418657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:54.429261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1185/2384] 10.222.0.57 () {58 vars in 967 bytes} [Wed Jun 15 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:54.459899 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:54.473772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae316a7-072b-4ae3-95cd-e433b588ac0a service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:54.482219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b7cec-fd8e-4c42-8355-5bcdba8d0438 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:54.483117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b7cec-fd8e-4c42-8355-5bcdba8d0438 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:54.483117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b7cec-fd8e-4c42-8355-5bcdba8d0438 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:54.483372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b7cec-fd8e-4c42-8355-5bcdba8d0438 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:54.508681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b7cec-fd8e-4c42-8355-5bcdba8d0438 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:54.511001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b7cec-fd8e-4c42-8355-5bcdba8d0438 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:54.520649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1200/2385] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:45:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:45:56.883919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:56.886057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:56.910155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53662-0757-4c38-97f0-67020b34b2d6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:56.916589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:56.917733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:56.918188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:56.918629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] PATH_INFO: `/v3/users/7d6eaf2c133743a08314ca0d9dcc59a8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:56.920017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6eaf2c133743a08314ca0d9dcc59a8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:56.922157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:56.982903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6eaf2c133743a08314ca0d9dcc59a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:56.996098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6eaf2c133743a08314ca0d9dcc59a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.003465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6eaf2c133743a08314ca0d9dcc59a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.013430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6eaf2c133743a08314ca0d9dcc59a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.014110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6eaf2c133743a08314ca0d9dcc59a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.022656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1186/2386] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:45:56 2022] DELETE /identity/v3/users/7d6eaf2c133743a08314ca0d9dcc59a8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:57.027662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.028117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.028777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.053972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:57.055236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:57.060176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:57.141226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1201/2387] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:57.152647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:57.154454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:57.236503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aff875-aef1-4825-9088-90042127432b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:57.241332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a49e65-3b18-4259-a44b-fa88c0f06d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:57.242055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a49e65-3b18-4259-a44b-fa88c0f06d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.242350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a49e65-3b18-4259-a44b-fa88c0f06d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.242627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a49e65-3b18-4259-a44b-fa88c0f06d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.261426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a49e65-3b18-4259-a44b-fa88c0f06d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:57.262929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a49e65-3b18-4259-a44b-fa88c0f06d91 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:57.296098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1187/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:57.576621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:57.579029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:57.600212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e838364-522e-4d1c-b8cc-7a3f64d9ad9e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:57.610839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:57.612821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.613279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.615544 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] PATH_INFO: `/v3/projects/a57bab77e60b4efc9df43e95f1ead38b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.630521 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a57bab77e60b4efc9df43e95f1ead38b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:57.632003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:57.702039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a57bab77e60b4efc9df43e95f1ead38b', 'request_id': 'req-e7fb1bee-6019-44bd-8eaf-98c532552c02'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.712388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a57bab77e60b4efc9df43e95f1ead38b', 'request_id': 'req-e7fb1bee-6019-44bd-8eaf-98c532552c02'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.719606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a57bab77e60b4efc9df43e95f1ead38b', 'request_id': 'req-e7fb1bee-6019-44bd-8eaf-98c532552c02'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.732707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] The token cache is being invalidate because project a57bab77e60b4efc9df43e95f1ead38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:45:57.733168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.733982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e7fb1bee-6019-44bd-8eaf-98c532552c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:45:57.735963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1202/2389] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:45:57 2022] DELETE /identity/v3/projects/a57bab77e60b4efc9df43e95f1ead38b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:57.741259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.741689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.742104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.763094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:57.764383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:57.821462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:57.842870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1188/2390] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:57.851620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.852014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.852207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.876955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:57.878227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:57.886397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1203/2391] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:57.890657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:57.902002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2e77-3a3b-4fff-b3bb-19c53eafbced None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:57.907586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:57.908277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.908569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.908853 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.909599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:45:57.909963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:57.911573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:57.929795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1189/2392] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:45:57.930053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:57.940684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a5df3-f524-4ca6-90be-b07057de66c6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:57.946952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:57.947770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:57.948240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:57.948618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:57.949611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:57.951827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:58.024263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1204/2393] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:57 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:58.027227 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:58.041880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a6d07-9483-404b-9190-4ec2113d5801 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:58.130246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:58.132739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.133191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.133699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.136356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:58.138920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:58.167889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1190/2394] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:58 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:58.172216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:58.187934 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381c0c-83a7-4d16-b304-2aa60f9797fb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:58.193417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:58.194581 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.195104 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.195617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.196566 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:58.199355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:58.214946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1205/2395] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:58.219772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:58.233027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b5f9-0a7d-49bf-a573-1c27a0e22b91 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:58.239801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08338b1b-95c7-49e3-b7ed-01630d4ed5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:58.240628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08338b1b-95c7-49e3-b7ed-01630d4ed5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.241595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08338b1b-95c7-49e3-b7ed-01630d4ed5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.241595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08338b1b-95c7-49e3-b7ed-01630d4ed5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.242231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08338b1b-95c7-49e3-b7ed-01630d4ed5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:58.244252 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08338b1b-95c7-49e3-b7ed-01630d4ed5b4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:58.260732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1191/2396] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:58.264822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:58.280487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfa789-31fc-4d43-b72d-60bc31c2606f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:58.285986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:58.287374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.287808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.289235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] PATH_INFO: `/v3/projects/79a3fa88f5ea4fa2940733b09cbc835a/users/af5c10fc72dc44e59030216239fc678d/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.351299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a3fa88f5ea4fa2940733b09cbc835a, user_id=af5c10fc72dc44e59030216239fc678d, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:58.357128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:58.391207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1206/2397] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:58 2022] PUT /identity/v3/projects/79a3fa88f5ea4fa2940733b09cbc835a/users/af5c10fc72dc44e59030216239fc678d/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:45:58.398386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b943e7-98eb-4532-b716-b0511129ff6d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.398671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b943e7-98eb-4532-b716-b0511129ff6d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.399005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b943e7-98eb-4532-b716-b0511129ff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.464195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-68b943e7-98eb-4532-b716-b0511129ff6d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:58.466214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-68b943e7-98eb-4532-b716-b0511129ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:58.608763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1192/2398] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:58.620450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:58.623102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:58.691686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:58.734252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ab5ed-d135-47e9-882a-f4e283271c3d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:58.741916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:58.743227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.743675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.744175 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.757130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:58.759616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:58.777851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1207/2399] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:45:58.884401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:58.885277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:58.885789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:58.920382 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] MFA Rules not processed for user `67d64cad336c44d28c9cf31fc9896854`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:45:58.922201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:58.969125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:45:58.997809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1193/2400] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:45:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:45:59.000973 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:59.002482 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:45:59.016249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b11cd-dbe9-4510-862f-09794dbb87d5 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:59.020864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:59.021527 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:59.021848 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:59.022127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:59.022882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:59.025336 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:59.060800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1208/2401] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:45:58 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:45:59.064956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:59.077222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b59a0d-b794-4859-9221-638f2797d2a4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:59.144408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:59.145238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:59.145806 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:59.146197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:59.147044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:59.149066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:59.177889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1194/2402] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:45:59 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:45:59.184062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:59.197781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b411fa9-5ed1-46f5-a980-125b4101fa94 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:59.204713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:59.205613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:59.206003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:59.206353 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:59.207199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:59.208889 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:59.223614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1209/2403] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:59.227962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:59.245940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862f9a4-931e-47b6-80d4-4a4b63b737bb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:59.253632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:59.254738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:59.255034 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:59.255319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] PATH_INFO: `/v3/projects/3e1f7cd040e246329d53abe443bb6b9a/users/d7775b4643f049808494ea2b0efecc6d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:59.275281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1f7cd040e246329d53abe443bb6b9a, user_id=d7775b4643f049808494ea2b0efecc6d, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:59.278098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:59.293090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1195/2404] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:59 2022] PUT /identity/v3/projects/3e1f7cd040e246329d53abe443bb6b9a/users/d7775b4643f049808494ea2b0efecc6d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:45:59.297802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:59.308712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccafe09-97e1-4f67-b38e-2ced974d8cc5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:59.378914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb84476-c4bf-4010-95f5-38d77de314af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:59.379743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb84476-c4bf-4010-95f5-38d77de314af admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:59.380101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb84476-c4bf-4010-95f5-38d77de314af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:59.380559 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb84476-c4bf-4010-95f5-38d77de314af admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:59.381485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb84476-c4bf-4010-95f5-38d77de314af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:59.383217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb84476-c4bf-4010-95f5-38d77de314af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:59.398192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1210/2405] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:45:59.400697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:45:59.411639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b713a1-3a9f-413d-89f6-37a5fbde570b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:45:59.416604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:45:59.417511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:45:59.417845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:45:59.418356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] PATH_INFO: `/v3/projects/3e1f7cd040e246329d53abe443bb6b9a/users/d7775b4643f049808494ea2b0efecc6d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:45:59.420381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1f7cd040e246329d53abe443bb6b9a, user_id=d7775b4643f049808494ea2b0efecc6d, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:45:59.423085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:45:59.438002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1196/2406] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:45:59 2022] PUT /identity/v3/projects/3e1f7cd040e246329d53abe443bb6b9a/users/d7775b4643f049808494ea2b0efecc6d/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:06.213027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:06.213788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:06.213788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:06.258313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] MFA Rules not processed for user `af5c10fc72dc44e59030216239fc678d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:06.260600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:06.455339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1211/2407] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:46:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:06.459841 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:06.646701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf867c-9844-4519-8575-c820881a8518 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:06.660653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:06.661369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:06.661887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:06.662179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:06.662918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:06.665360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:06.749521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1197/2408] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:46:06 2022] POST /identity/v3/projects => generated 371 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:46:06.753311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:06.768695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7262391-376a-45f2-81d8-549b6e0c4962 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:06.809265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:06.874134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:06.875308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:06.875813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:06.876478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:06.877866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:06.881623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:06.920379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1212/2409] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:46:06 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:46:06.925472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:06.944491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c1dcc-3433-4127-bf05-530bf4deb645 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:06.950813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:06.951556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:06.951900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:06.952189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:06.953044 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:06.955006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:06.966885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1198/2410] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:46:06.970835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:06.983271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a260e-133e-4b8d-9de8-77275f03db58 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:06.988673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:06.989579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:06.990066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:06.990363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] PATH_INFO: `/v3/projects/fab17e2c4af24819ad1de1b83ef2e464/users/9a8f02c2087d4048a8f6f3a0d91f3eca/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:07.017007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab17e2c4af24819ad1de1b83ef2e464, user_id=9a8f02c2087d4048a8f6f3a0d91f3eca, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:07.023362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:07.054375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1213/2411] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:46:06 2022] PUT /identity/v3/projects/fab17e2c4af24819ad1de1b83ef2e464/users/9a8f02c2087d4048a8f6f3a0d91f3eca/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:46:07.056692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:07.072457 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a16ece-bab0-4026-af65-44887ed42405 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:07.079407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:07.197014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cea8c-d16e-4587-998c-91ad9a843da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:07.197758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cea8c-d16e-4587-998c-91ad9a843da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:07.198050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cea8c-d16e-4587-998c-91ad9a843da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:07.198333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cea8c-d16e-4587-998c-91ad9a843da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:07.199086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cea8c-d16e-4587-998c-91ad9a843da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:07.201332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cea8c-d16e-4587-998c-91ad9a843da1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:07.235215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:07.236337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1199/2412] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:46:07.248233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ac07c-063f-4dd1-8a88-03f3c6624e01 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:07.253770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:07.255223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:07.255626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:07.256154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] PATH_INFO: `/v3/projects/fab17e2c4af24819ad1de1b83ef2e464/users/9a8f02c2087d4048a8f6f3a0d91f3eca/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:07.260719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab17e2c4af24819ad1de1b83ef2e464, user_id=9a8f02c2087d4048a8f6f3a0d91f3eca, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:07.263680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:07.283026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1214/2413] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:46:07 2022] PUT /identity/v3/projects/fab17e2c4af24819ad1de1b83ef2e464/users/9a8f02c2087d4048a8f6f3a0d91f3eca/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:46:07.701510 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c5105-0613-40b8-aa90-4b0f66abdbed None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:07.702009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c5105-0613-40b8-aa90-4b0f66abdbed None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:07.702131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c5105-0613-40b8-aa90-4b0f66abdbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:07.764077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-833c5105-0613-40b8-aa90-4b0f66abdbed None None] MFA Rules not processed for user `d7775b4643f049808494ea2b0efecc6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:07.765330 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-833c5105-0613-40b8-aa90-4b0f66abdbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:07.902834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1200/2414] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:46:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:08.317341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:08.319562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:08.627546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bfca8-2413-4275-9a2f-c086a82c68e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:08.648159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9324c-bd07-4a39-8a8d-4244408c55a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:08.649078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9324c-bd07-4a39-8a8d-4244408c55a7 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:08.650019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9324c-bd07-4a39-8a8d-4244408c55a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:08.650414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9324c-bd07-4a39-8a8d-4244408c55a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:08.753444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:08.761919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9324c-bd07-4a39-8a8d-4244408c55a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:08.764539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9324c-bd07-4a39-8a8d-4244408c55a7 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:08.792670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1215/2415] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:14.717808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.718968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.719391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.729457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:14.730626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:14.731421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:14.780803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] MFA Rules not processed for user `9a8f02c2087d4048a8f6f3a0d91f3eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:14.782092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:14.835112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:14.836493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:14.948872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1201/2416] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:14.953168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1216/2417] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:46:14.963151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:14.971490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:14.994179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:15.011590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f0e66-8d64-4552-bf87-b754584323f4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:15.018093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de74cf-8991-47dd-b8ed-e52e76234d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:15.018779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de74cf-8991-47dd-b8ed-e52e76234d97 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:15.019074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de74cf-8991-47dd-b8ed-e52e76234d97 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:15.019361 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de74cf-8991-47dd-b8ed-e52e76234d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:15.021179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:15.106936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de74cf-8991-47dd-b8ed-e52e76234d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:15.109262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de74cf-8991-47dd-b8ed-e52e76234d97 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:15.129308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1217/2418] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:46:15.178653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1dfa1-a2c8-4dd8-a2a9-9c3b887538de None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:15.191574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a2d8f-4b27-4f8b-b607-90df1f59aca9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:15.192334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a2d8f-4b27-4f8b-b607-90df1f59aca9 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:15.192723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a2d8f-4b27-4f8b-b607-90df1f59aca9 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:15.193107 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a2d8f-4b27-4f8b-b607-90df1f59aca9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:15.203939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a2d8f-4b27-4f8b-b607-90df1f59aca9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:15.205831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a2d8f-4b27-4f8b-b607-90df1f59aca9 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:15.216766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1202/2419] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:46:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:46:15.798187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:15.798676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:15.798768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:15.821875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] MFA Rules not processed for user `ed7372fed3e14f3e98ca25d22a7c7a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:15.823681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:15.953884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1218/2420] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:46:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:16.973937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:16.974922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:16.975383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:17.002607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] MFA Rules not processed for user `af5c10fc72dc44e59030216239fc678d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:17.006058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:17.014951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1203/2421] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:46:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:25.945775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:26.995223 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=37 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:31.193827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:31.195248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:31.211014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:31.212431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:31.298256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e1581-b0fd-4eba-9b4f-a35c9a837958 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:31.300432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2223ee-750a-4149-997c-86751a28d5e6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:31.303709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:31.304580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:31.304840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:31.305984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:31.308107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:31.308234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:31.308833 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:31.309063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:31.309354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:31.311190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:31.320541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:31.322201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34055ea9-b171-4ced-8525-a5090c342eff service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:31.324368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:31.327308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04a6fb-6ad8-4f0e-85c5-fa9d4c3209d3 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:31.336822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1219/2422] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:31.339078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1204/2423] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:46:32.813734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb8746-3089-4512-baa8-2fb36579242e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:32.814240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb8746-3089-4512-baa8-2fb36579242e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:32.814333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb8746-3089-4512-baa8-2fb36579242e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:32.815701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1220/2424] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:46:32.822077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488846f1-d14e-4824-a8f0-aefdebf71f2d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:32.823170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488846f1-d14e-4824-a8f0-aefdebf71f2d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:32.823170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488846f1-d14e-4824-a8f0-aefdebf71f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:32.869762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-488846f1-d14e-4824-a8f0-aefdebf71f2d None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:32.870910 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-488846f1-d14e-4824-a8f0-aefdebf71f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:32.880804 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1205/2425] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:46:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:32.892422 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb8746-3089-4512-baa8-2fb36579242e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:32.894193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5bcb8746-3089-4512-baa8-2fb36579242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:32.983660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb8746-3089-4512-baa8-2fb36579242e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:32.987168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac147e-bdfe-483f-8fcd-8f01b78604be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:32.987818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac147e-bdfe-483f-8fcd-8f01b78604be service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:32.988108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac147e-bdfe-483f-8fcd-8f01b78604be service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:32.988393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac147e-bdfe-483f-8fcd-8f01b78604be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:32.998596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac147e-bdfe-483f-8fcd-8f01b78604be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:33.000022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac147e-bdfe-483f-8fcd-8f01b78604be service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:33.011131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1221/2426] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:46:33.077573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dce5d0-d533-433f-b00c-5c61e3d7e4b8 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:33.077870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dce5d0-d533-433f-b00c-5c61e3d7e4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:33.078156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dce5d0-d533-433f-b00c-5c61e3d7e4b8 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:33.080313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1206/2427] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:46:33.084516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93117b0-d49b-490f-b986-1e578309e42f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:33.084814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93117b0-d49b-490f-b986-1e578309e42f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:33.085092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93117b0-d49b-490f-b986-1e578309e42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:33.105163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b93117b0-d49b-490f-b986-1e578309e42f None None] MFA Rules not processed for user `d292189b741b40b8859bfe9883b4c5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:33.106441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b93117b0-d49b-490f-b986-1e578309e42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:33.162648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1222/2428] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:46:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:33.174977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dce5d0-d533-433f-b00c-5c61e3d7e4b8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:33.176903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-38dce5d0-d533-433f-b00c-5c61e3d7e4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:33.193171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dce5d0-d533-433f-b00c-5c61e3d7e4b8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:33.198415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edb7f7-fe98-4b36-83dd-31dbcb946a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:33.198814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edb7f7-fe98-4b36-83dd-31dbcb946a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:33.199100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edb7f7-fe98-4b36-83dd-31dbcb946a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:33.199378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edb7f7-fe98-4b36-83dd-31dbcb946a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:33.207198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edb7f7-fe98-4b36-83dd-31dbcb946a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:33.208643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edb7f7-fe98-4b36-83dd-31dbcb946a44 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:33.219883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1207/2429] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:33.325484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e3924-1f94-4918-9197-63979781895f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:33.325829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e3924-1f94-4918-9197-63979781895f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:33.326122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e3924-1f94-4918-9197-63979781895f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:33.327510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1223/2430] 10.222.0.57 () {54 vars in 921 bytes} [Wed Jun 15 14:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:46:33.332755 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d9ad-d8b8-4845-8ad7-2b6d549bbd01 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:33.333060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d9ad-d8b8-4845-8ad7-2b6d549bbd01 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:33.333369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d9ad-d8b8-4845-8ad7-2b6d549bbd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:33.353316 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f810d9ad-d8b8-4845-8ad7-2b6d549bbd01 None None] MFA Rules not processed for user `f5953aa40eb54d399b593000f3c74596`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:33.354599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f810d9ad-d8b8-4845-8ad7-2b6d549bbd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:33.363970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1208/2431] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Jun 15 14:46:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:33.375101 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e3924-1f94-4918-9197-63979781895f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:33.380499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:33.388359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e3924-1f94-4918-9197-63979781895f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:33.391894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d838c91-4192-4a88-af14-2ccf0bc0d2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:33.392532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d838c91-4192-4a88-af14-2ccf0bc0d2db service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:33.392818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d838c91-4192-4a88-af14-2ccf0bc0d2db service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:33.393095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d838c91-4192-4a88-af14-2ccf0bc0d2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:33.403998 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d838c91-4192-4a88-af14-2ccf0bc0d2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:33.405519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d838c91-4192-4a88-af14-2ccf0bc0d2db service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:33.416995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1224/2432] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:46:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:46:36.979673 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:36.980430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:36.980430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:36.998401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] MFA Rules not processed for user `67d64cad336c44d28c9cf31fc9896854`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:36.999646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:37.006603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1209/2433] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:46:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:40.740473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77821492-218b-4463-8eb6-d8900296d051 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:40.741029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77821492-218b-4463-8eb6-d8900296d051 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:40.741029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77821492-218b-4463-8eb6-d8900296d051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:40.764981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-77821492-218b-4463-8eb6-d8900296d051 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:46:40.766564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77821492-218b-4463-8eb6-d8900296d051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:46:40.779756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1225/2434] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:46:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:46:40.791148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:46:40.804268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52c9b5-5668-4478-ad78-4276075d613f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:46:40.810859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4693110-423e-4f5f-9bd0-3dd8eafc6dfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:46:40.811696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4693110-423e-4f5f-9bd0-3dd8eafc6dfc service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:46:40.811985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4693110-423e-4f5f-9bd0-3dd8eafc6dfc service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:46:40.812320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4693110-423e-4f5f-9bd0-3dd8eafc6dfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:46:40.830488 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4693110-423e-4f5f-9bd0-3dd8eafc6dfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:46:40.833422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4693110-423e-4f5f-9bd0-3dd8eafc6dfc service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:46:40.845751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1210/2435] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:46:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:46:50.757386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:46:50.840758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=63 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:13.778846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77821492-218b-4463-8eb6-d8900296d051 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:13.788225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77821492-218b-4463-8eb6-d8900296d051 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:13.791807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:13.792478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:13.792768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:13.793050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:13.802431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:13.803859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:13.814051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1226/2436] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:19.715070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:19.715603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:19.715705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:19.733965 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:19.736050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:19.743345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1211/2437] 10.222.0.57 () {58 vars in 967 bytes} [Wed Jun 15 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:19.764708 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:19.781404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d063f81-e481-402a-8231-8fe305417eda service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:19.787374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:19.788037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:19.788325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:19.788607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:19.797925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:19.799498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:19.807510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1227/2438] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:47:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:47:23.322951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:23.333305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057648c-a787-44b2-bc20-c589dccc68d9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:23.336929 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:23.337576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:23.337895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:23.338177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:23.348637 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:23.350112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:23.360274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1212/2439] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:29.804250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=12 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:33.355448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=9 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:02.845006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:02.855427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f0ac-eaad-4898-a12e-3d22334e7ef9 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:02.859057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:02.859702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:02.859990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:02.860272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:02.862127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:02.884341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:02.885831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70df7c-652b-4ecf-920c-4193d26b4e84 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:02.897631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1228/2440] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:48:04.066756 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:04.076097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098dde4-f4d5-4fd3-b6cf-cdf7a13d9bbf service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:04.079798 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:04.080440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:04.080733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:04.081023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:04.082824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:04.096455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:04.097896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:04.108663 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1213/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:12.892286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:14.103872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:23.453607 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.454095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.454198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.475792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:23.476761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:23.484831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1229/2442] 10.222.0.57 () {58 vars in 967 bytes} [Wed Jun 15 14:48:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:23.533904 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:23.546074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0e67c-b642-4bfe-be7b-34449dd4adfa service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:23.551486 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:23.552247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.552688 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.553120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.561842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:23.563416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:23.571625 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1214/2443] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:48:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:48:23.711083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:23.727834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c2ac5-3f44-4e87-a5d7-d6673bafe5e5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:23.735049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:23.735844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.736214 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.736500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.738344 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:23.750009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:23.751602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:23.762283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1230/2444] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:23.794076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:23.806752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7da10-131e-4dc3-9ea7-af6cc750fd6b service swift] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:23.810446 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:23.811158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.811454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.811740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.813597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:23.825058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:23.826561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:23.838417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1215/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:30.675152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:30.689091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96719-7ae8-4d89-a46d-d06cdb45680b service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:30.696309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:30.697476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:30.697802 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:30.698183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:30.699999 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:30.715692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:30.717297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:30.729223 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1231/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:30.748489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:30.750413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:30.831149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11937b-5821-47af-aae4-b0be738accd9 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:30.835790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:30.836664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:30.836952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:30.837232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] PATH_INFO: `/v3/users/163fb6502e0442c7aefc0c122ed4c035` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:30.839072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] RBAC: Authorizing `identity:delete_user(user_id=163fb6502e0442c7aefc0c122ed4c035)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:30.841019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:30.887620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163fb6502e0442c7aefc0c122ed4c035'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:30.899285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163fb6502e0442c7aefc0c122ed4c035'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:30.906178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163fb6502e0442c7aefc0c122ed4c035'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:30.913340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163fb6502e0442c7aefc0c122ed4c035'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:30.913982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163fb6502e0442c7aefc0c122ed4c035'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:30.920323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1216/2447] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:48:30 2022] DELETE /identity/v3/users/163fb6502e0442c7aefc0c122ed4c035 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:48:31.091680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:31.106660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff57c3-740c-4aae-8802-b6e60c4589b6 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:31.164662 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:31.165543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:31.165939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:31.166297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] PATH_INFO: `/v3/projects/f6fb5fc3024f428d9d052371dbe90737` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:31.167511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fb5fc3024f428d9d052371dbe90737)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:31.169406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:31.214646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fb5fc3024f428d9d052371dbe90737', 'request_id': 'req-4213cd0a-9366-4768-9b77-9f4e36707e6f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.221920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fb5fc3024f428d9d052371dbe90737', 'request_id': 'req-4213cd0a-9366-4768-9b77-9f4e36707e6f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.228810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fb5fc3024f428d9d052371dbe90737', 'request_id': 'req-4213cd0a-9366-4768-9b77-9f4e36707e6f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.240008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] The token cache is being invalidate because project f6fb5fc3024f428d9d052371dbe90737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:48:31.240302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.240956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.242648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1232/2448] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:48:31 2022] DELETE /identity/v3/projects/f6fb5fc3024f428d9d052371dbe90737 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:48:31.247285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:31.248917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:31.264552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce65e6-19eb-4712-9bed-b8f1922d333e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:31.325183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:31.337140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:31.338100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:31.338485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:31.338895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] PATH_INFO: `/v3/users/b25f8f68987848d7864fd0b0119cf86a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:31.340214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b25f8f68987848d7864fd0b0119cf86a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:31.342173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:31.398015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b25f8f68987848d7864fd0b0119cf86a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.413842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b25f8f68987848d7864fd0b0119cf86a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.421513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b25f8f68987848d7864fd0b0119cf86a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.429974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b25f8f68987848d7864fd0b0119cf86a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.430757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b25f8f68987848d7864fd0b0119cf86a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.437970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1217/2449] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:48:31 2022] DELETE /identity/v3/users/b25f8f68987848d7864fd0b0119cf86a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:48:31.611654 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:31.613227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:31.629729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213cd0a-9366-4768-9b77-9f4e36707e6f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:31.696438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:31.703945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:31.704931 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:31.705396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:31.705878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] PATH_INFO: `/v3/projects/c6754cfb3b88408eb39872bbbf457d19` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:31.707309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6754cfb3b88408eb39872bbbf457d19)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:31.709202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:31.755900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6754cfb3b88408eb39872bbbf457d19', 'request_id': 'req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.763364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6754cfb3b88408eb39872bbbf457d19', 'request_id': 'req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.768833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6754cfb3b88408eb39872bbbf457d19', 'request_id': 'req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.780208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] The token cache is being invalidate because project c6754cfb3b88408eb39872bbbf457d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:48:31.780501 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.781139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:48:31.782713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1233/2450] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:48:31 2022] DELETE /identity/v3/projects/c6754cfb3b88408eb39872bbbf457d19 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:48:32.204134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:32.205566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:32.280008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e71e5-47b8-4063-b8bc-467556eeccd4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:32.344547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:32.345469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:32.345790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:32.346073 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:32.347830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:32.354005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:32.407139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:32.408712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59159139-4a62-4b44-8431-fbf6b09a0009 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:32.418266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1218/2451] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:39.038577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:39.040542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:39.058225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fde1f-e2a4-4752-9e06-0fbc30d5bb37 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:39.064327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:39.064995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:39.065282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:39.065562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:39.067852 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:39.082687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:39.084277 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:39.097849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1234/2452] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:41.810792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:41.811274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:41.811415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:41.828845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] MFA Rules not processed for user `1d58ac7f49454fd6a5b5eef6a1e8b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:41.830090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:41.837164 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1219/2453] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:42.144608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:42.146126 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:42.204543 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bba5b-b3f1-47a0-8a9b-3578a0f7bb6a service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:42.208125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:42.208778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:42.209067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:42.209345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:42.211163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:42.221256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:42.222720 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a313d-5600-4e9c-81c8-5af50975a40f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:42.231040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1235/2454] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:42.543199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:42.544589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:42.614973 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd15c1-701e-497a-85bb-ae690fc62d76 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:42.620786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:42.621431 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:42.621744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:42.622028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:42.623791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:42.635159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:42.636710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:42.646387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1220/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:49.574840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:49.575340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:49.575340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:49.593996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] MFA Rules not processed for user `ed7372fed3e14f3e98ca25d22a7c7a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:49.595621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:49.654520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1236/2456] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:52.642307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=59 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:54.045740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:54.059727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6c74-5a21-4584-ab04-1150af327f30 service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:54.065245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:54.065944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:54.066234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:54.066516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:54.068292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:54.081331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:54.082967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:54.097108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1221/2457] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:54.554914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:54.568300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9891421-3c68-4d06-bc84-4110bb7ae4fc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:54.576425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:54.578144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:54.578657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:54.580340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:54.582979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:54.599168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:54.601167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5252b34-44d4-4bf0-900c-6d39916d7684 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:54.613010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1237/2458] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:04.090093 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:04.608252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:07.865463 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:07.866875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:07.923305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5884d-e676-43ab-aa48-6e3c2c7a9fff service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:07.926993 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:07.927613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:07.927917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:07.928187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:07.929982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:07.995829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:07.997310 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:08.006540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1222/2459] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:14.744645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6eea-58a5-4323-aa6c-276e07d807c6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:14.745137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6eea-58a5-4323-aa6c-276e07d807c6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:14.745283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6eea-58a5-4323-aa6c-276e07d807c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:14.762979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-752c6eea-58a5-4323-aa6c-276e07d807c6 None None] MFA Rules not processed for user `28ff37865a9b49ec8601140fca00a627`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:14.764179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-752c6eea-58a5-4323-aa6c-276e07d807c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:14.771709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1238/2460] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:49:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:15.069799 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:15.071211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:15.083362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960cd94-4fea-41a6-b4e6-e9b5ba6f1db6 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:15.086952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:15.087693 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:15.088642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:15.089009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:15.091007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:15.101123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:15.102556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:15.110766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1223/2461] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:24.757218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:25.107236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=54 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:56.506892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e757a7-17e5-4f90-affb-c71538ff6508 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:56.507489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e757a7-17e5-4f90-affb-c71538ff6508 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:56.507489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e757a7-17e5-4f90-affb-c71538ff6508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:56.525094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-27e757a7-17e5-4f90-affb-c71538ff6508 None None] MFA Rules not processed for user `67d64cad336c44d28c9cf31fc9896854`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:56.526410 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-27e757a7-17e5-4f90-affb-c71538ff6508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:56.580916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1239/2462] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:06.574075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=25 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:08.494423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:08.509274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19b99-4bae-4cf8-9e77-6316eb5d9735 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.513143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.513921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.514278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.514672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.516747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:08.581110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.582576 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.592160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1224/2463] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:10.118862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.119796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.120255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.142120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:10.143925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:10.217430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1240/2464] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:10.272466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:10.274046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:10.363413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de0b1-0d15-414f-aa46-be010be7448b service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:10.372075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:10.372571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:10.373143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:10.373694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:10.384971 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:10.387035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:10.402423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1225/2465] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:50:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:50:12.638041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:12.641283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:12.657839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a1dab-73a8-4a06-b56f-1c8e95cd42fd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:12.663024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:12.664197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:12.664703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:12.665192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] PATH_INFO: `/v3/users/ed7372fed3e14f3e98ca25d22a7c7a7e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:12.666825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7372fed3e14f3e98ca25d22a7c7a7e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:12.669038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:12.732456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7372fed3e14f3e98ca25d22a7c7a7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:12.734190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7372fed3e14f3e98ca25d22a7c7a7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:12.747028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7372fed3e14f3e98ca25d22a7c7a7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:12.748388 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7372fed3e14f3e98ca25d22a7c7a7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:12.765037 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95b03532-c9d8-42b3-bb9d-81ab6d45e639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7372fed3e14f3e98ca25d22a7c7a7e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:12.785623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1241/2466] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:50:12 2022] DELETE /identity/v3/users/ed7372fed3e14f3e98ca25d22a7c7a7e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:13.031242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.045784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863fb6-4360-4f8a-b397-5306b7fd7637 service swift] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.134505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.135659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.136009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.136294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] PATH_INFO: `/v3/projects/26d36b3b04f046d3bee421c5987b965b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.137482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d36b3b04f046d3bee421c5987b965b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.140376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.180685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:13.206969 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d36b3b04f046d3bee421c5987b965b', 'request_id': 'req-fecd7c88-a00b-46e6-9633-5d8459604bfc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:13.213588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d36b3b04f046d3bee421c5987b965b', 'request_id': 'req-fecd7c88-a00b-46e6-9633-5d8459604bfc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:13.223415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d36b3b04f046d3bee421c5987b965b', 'request_id': 'req-fecd7c88-a00b-46e6-9633-5d8459604bfc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:13.247345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] The token cache is being invalidate because project 26d36b3b04f046d3bee421c5987b965b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:50:13.247670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:13.248336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fecd7c88-a00b-46e6-9633-5d8459604bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:50:13.253442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1226/2467] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:50:13 2022] DELETE /identity/v3/projects/26d36b3b04f046d3bee421c5987b965b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:50:13.258578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.258878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.259163 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.282411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:13.284275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:13.396347 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1242/2468] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:13.404166 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.404696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.405163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.427661 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:13.428997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:13.439050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1227/2469] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:13.443071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.454548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1368c3-91b4-4d03-ad8a-eeb3c26db016 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.459661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.460337 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.460627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.460905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.461649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:50:13.461989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.463608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.481264 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1243/2470] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:50:13.481876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.507348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b7e1-83aa-4596-8ac1-18c833d1d0d0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.519171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.519875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.528443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.528443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.528443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.531526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.581817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1228/2471] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:50:13.585693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.593775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:13.598202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fd19c-dc15-4068-8e90-d9bd85418aad admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.664384 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.665875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.666230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.666580 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.667365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.669398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.697006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1244/2472] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:50:13.701203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.713742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3fc3c-252a-444f-be9b-f555bcbbf6fd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.719485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.720903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.721213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.721492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.722271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.724718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.742669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1229/2473] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:13.743212 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.754240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded3e10-9f75-4472-a022-d27dc4c39a0a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.759869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d436aa-2f50-4795-8d16-106b6cae3012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.760691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d436aa-2f50-4795-8d16-106b6cae3012 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.761041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d436aa-2f50-4795-8d16-106b6cae3012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.761392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d436aa-2f50-4795-8d16-106b6cae3012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.762231 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d436aa-2f50-4795-8d16-106b6cae3012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.763927 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d436aa-2f50-4795-8d16-106b6cae3012 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.802575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1245/2474] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:13.806701 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:13.834890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5db70-383d-4bb6-a4f2-17105d51f139 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:13.845737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:13.846674 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.846962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.847277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] PATH_INFO: `/v3/projects/591d111357224477998b265d45e3ee0d/users/6caa64aecb914e968d057b6b593f667f/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.879230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] RBAC: Authorizing `identity:create_grant(project_id=591d111357224477998b265d45e3ee0d, user_id=6caa64aecb914e968d057b6b593f667f, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:13.882513 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:13.915887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1230/2475] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:50:13 2022] PUT /identity/v3/projects/591d111357224477998b265d45e3ee0d/users/6caa64aecb914e968d057b6b593f667f/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:50:13.917333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5a78c-a7c1-47fd-b220-5270e16635c5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:13.917680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5a78c-a7c1-47fd-b220-5270e16635c5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:13.917958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5a78c-a7c1-47fd-b220-5270e16635c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:13.940308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-77b5a78c-a7c1-47fd-b220-5270e16635c5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:13.941707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77b5a78c-a7c1-47fd-b220-5270e16635c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:14.024853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1246/2476] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:14.035496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:14.037264 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:14.101808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db219d65-6ad8-42b5-8304-c65ef590373e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:14.106266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:14.107668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:14.110522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:14.110522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:14.137594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:14.139079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:14.161302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1231/2477] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:21.381252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3bfc4-88d7-4d3c-a65b-2647cd91693b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.381892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3bfc4-88d7-4d3c-a65b-2647cd91693b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.381892 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3bfc4-88d7-4d3c-a65b-2647cd91693b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.462462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d1b3bfc4-88d7-4d3c-a65b-2647cd91693b None None] MFA Rules not processed for user `6caa64aecb914e968d057b6b593f667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:21.463764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d1b3bfc4-88d7-4d3c-a65b-2647cd91693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.607977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1247/2478] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:21.718311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:21.723779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:21.838868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:21.916080 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea9a4b-a445-4610-957a-2b81342828bc service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:21.926835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae4b2c-4c44-41fc-b7f4-b9882c6ae565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:21.930002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae4b2c-4c44-41fc-b7f4-b9882c6ae565 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:21.930526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae4b2c-4c44-41fc-b7f4-b9882c6ae565 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:21.930977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae4b2c-4c44-41fc-b7f4-b9882c6ae565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:21.943103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae4b2c-4c44-41fc-b7f4-b9882c6ae565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:21.944980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae4b2c-4c44-41fc-b7f4-b9882c6ae565 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:21.958225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1232/2479] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:24.852636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:24.853145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:24.853228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:24.889547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] MFA Rules not processed for user `6caa64aecb914e968d057b6b593f667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:24.891890 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:24.912429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1248/2480] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:50:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:31.951782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=14 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:34.872942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:36.839914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3150a-5886-4885-ac29-32bf0eb1113a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:36.840400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3150a-5886-4885-ac29-32bf0eb1113a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:36.840565 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3150a-5886-4885-ac29-32bf0eb1113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:36.858587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-92d3150a-5886-4885-ac29-32bf0eb1113a None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:36.859849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-92d3150a-5886-4885-ac29-32bf0eb1113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:36.914182 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1233/2481] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:36.926433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:36.927948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:36.998475 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae7e2c-3cad-40c7-8647-55fcb043f743 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:37.003584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:37.004266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:37.004555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:37.004834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:37.013474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:37.015605 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:37.023983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1249/2482] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:50:46.907539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=25 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:47.020573 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=32 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:57.515313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:57.515824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:57.515964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:57.534778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:57.535985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:57.543169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1234/2483] 10.222.0.57 () {58 vars in 967 bytes} [Wed Jun 15 14:50:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:57.557252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:57.572550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7dce0-ce61-4062-8ee7-d3089a3c940e service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:57.581312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:57.582199 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:57.582496 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:57.582805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:57.593358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:57.595014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:57.605681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1250/2484] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:50:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:51:07.528720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:07.601415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:16.455121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:16.457225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.468667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da9640-5da4-4a0a-b6ae-f5ca967f6737 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:16.472301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:16.472949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:16.473244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:16.473529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:16.475302 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:16.487646 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:16.489067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:16.499058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1235/2485] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:19.270443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.272024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:19.332082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.334217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:19.357493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bffe7-6b30-46d9-837b-0669e319b5a9 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.365992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.367696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.368360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.368866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] PATH_INFO: `/v3/users/28ff37865a9b49ec8601140fca00a627` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.370034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9392c-b975-44b5-a9de-2d45d43e5b80 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.371024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ff37865a9b49ec8601140fca00a627)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.375063 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:19.375529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.376416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.376649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.376929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.379937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:19.393777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.396249 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:19.410455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1251/2486] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:19.477405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ff37865a9b49ec8601140fca00a627'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.498505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ff37865a9b49ec8601140fca00a627'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.507638 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ff37865a9b49ec8601140fca00a627'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.522582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ff37865a9b49ec8601140fca00a627'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.523664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c0da8697-1fe1-45a7-b6ec-ed8ae25807ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ff37865a9b49ec8601140fca00a627'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.531848 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1236/2487] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:19 2022] DELETE /identity/v3/users/28ff37865a9b49ec8601140fca00a627 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:19.725263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:19.738664 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e180a-3fbc-4565-85b8-432afe3b20a0 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:19.796424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:19.797305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.797587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.797895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] PATH_INFO: `/v3/projects/0c12c8e1398843eca8746543a0684841` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.799615 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c12c8e1398843eca8746543a0684841)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:19.801552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:19.849594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c12c8e1398843eca8746543a0684841', 'request_id': 'req-a8e440b6-c63f-4d45-a5bf-309d04c08af9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.856940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c12c8e1398843eca8746543a0684841', 'request_id': 'req-a8e440b6-c63f-4d45-a5bf-309d04c08af9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.862533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c12c8e1398843eca8746543a0684841', 'request_id': 'req-a8e440b6-c63f-4d45-a5bf-309d04c08af9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.873524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] The token cache is being invalidate because project 0c12c8e1398843eca8746543a0684841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:19.873842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.874490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a8e440b6-c63f-4d45-a5bf-309d04c08af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:19.876033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1252/2488] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:19 2022] DELETE /identity/v3/projects/0c12c8e1398843eca8746543a0684841 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:19.881555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.881889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.882206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.900757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:19.902028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:19.961110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1237/2489] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:19.969216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:19.969515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:19.969856 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:19.988309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:19.989522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:19.997469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1253/2490] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:20.001616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.012277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e828f-4acb-4b41-b6f2-0c638a47a384 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.016917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.017571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.018422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.019293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.020023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:20.020338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.021949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.033376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1238/2491] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:20.037824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.046901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c622f-46b0-4649-baf8-970e9694ff69 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.051900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.052550 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.052832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.053106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.053840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.055722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.066379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:20.084195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1254/2492] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:20 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:20.088255 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.098320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242e0e2-ac0a-401b-bf83-3a0762f56309 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.142547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:20.157129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.157872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.158161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.158447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.159469 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.161477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.180768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1239/2493] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:20 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:20.184863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.195785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bbc1e-54e1-4a73-a413-2c419ae34dd4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.200442 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.201106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.201400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.201707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.202439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.204041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.215008 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1255/2494] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:20.218842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.228888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d27b4-8d88-4f84-abe6-bfd78e23efdb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.233526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0fb93-8be4-474e-bff7-db8cf3f01700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.234213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0fb93-8be4-474e-bff7-db8cf3f01700 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.234504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0fb93-8be4-474e-bff7-db8cf3f01700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.234784 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0fb93-8be4-474e-bff7-db8cf3f01700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.235506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0fb93-8be4-474e-bff7-db8cf3f01700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.237645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0fb93-8be4-474e-bff7-db8cf3f01700 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.247518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1240/2495] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:20.251414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.262502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85f9c1-5a83-417a-be41-7a6010837ded admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.267153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.268165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.268458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.268736 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] PATH_INFO: `/v3/projects/36ce2c1d9a9f4e5d95e0276769754b03/users/35f264402d884087ac9582ba4791a40b/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.283354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ce2c1d9a9f4e5d95e0276769754b03, user_id=35f264402d884087ac9582ba4791a40b, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.286192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.300358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1256/2496] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:20 2022] PUT /identity/v3/projects/36ce2c1d9a9f4e5d95e0276769754b03/users/35f264402d884087ac9582ba4791a40b/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:20.311079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.311376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.311668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.341022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] MFA Rules not processed for user `35f264402d884087ac9582ba4791a40b`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:20.342284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:20.412703 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1241/2497] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:20.424872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.426641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:20.500516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8309-9409-4951-ac12-a098a5ca9815 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.506413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b7add-dca4-4db6-9c79-224b85b9161e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.507073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b7add-dca4-4db6-9c79-224b85b9161e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.507365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b7add-dca4-4db6-9c79-224b85b9161e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.507648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b7add-dca4-4db6-9c79-224b85b9161e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.515532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b7add-dca4-4db6-9c79-224b85b9161e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.517138 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b7add-dca4-4db6-9c79-224b85b9161e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.526915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1257/2498] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:20.741850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.803614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f928b1-a0a6-4d07-9572-d20a23d574c4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:20.808071 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:20.808954 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.809305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.809990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] PATH_INFO: `/v3/users/35f264402d884087ac9582ba4791a40b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.810706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f264402d884087ac9582ba4791a40b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:20.812617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:20.860309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f264402d884087ac9582ba4791a40b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.867526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:20.874356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f264402d884087ac9582ba4791a40b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.882470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f264402d884087ac9582ba4791a40b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.889649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f264402d884087ac9582ba4791a40b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.890290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f264402d884087ac9582ba4791a40b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:20.897199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1242/2499] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:20 2022] DELETE /identity/v3/users/35f264402d884087ac9582ba4791a40b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:20.901703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:20.902007 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:20.902324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:20.919105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:20.920242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:20.978089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1258/2500] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:20.988534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:20.990078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.046306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee91dc5-b722-47e0-9552-08618f1b37cd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.050587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93252354-34c6-4275-83f5-113eb3abf884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.051229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93252354-34c6-4275-83f5-113eb3abf884 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.051518 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93252354-34c6-4275-83f5-113eb3abf884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.051801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93252354-34c6-4275-83f5-113eb3abf884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.060723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93252354-34c6-4275-83f5-113eb3abf884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.062173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93252354-34c6-4275-83f5-113eb3abf884 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.072338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1243/2501] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:21.336275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.337885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.354870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b39514-fc09-42f8-8bb1-2474955eb914 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.359809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.360959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.361151 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.361463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] PATH_INFO: `/v3/projects/36ce2c1d9a9f4e5d95e0276769754b03` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.362805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ce2c1d9a9f4e5d95e0276769754b03)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.365379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.405029 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:21.415569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ce2c1d9a9f4e5d95e0276769754b03', 'request_id': 'req-e9d4e7ce-8155-433f-b746-7474972b18f6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.422651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ce2c1d9a9f4e5d95e0276769754b03', 'request_id': 'req-e9d4e7ce-8155-433f-b746-7474972b18f6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.428211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ce2c1d9a9f4e5d95e0276769754b03', 'request_id': 'req-e9d4e7ce-8155-433f-b746-7474972b18f6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.439105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] The token cache is being invalidate because project 36ce2c1d9a9f4e5d95e0276769754b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:21.439399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.440027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e9d4e7ce-8155-433f-b746-7474972b18f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:21.442185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1259/2502] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:21 2022] DELETE /identity/v3/projects/36ce2c1d9a9f4e5d95e0276769754b03 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:21.449534 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.450047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.450151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.468430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:21.469573 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.529134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1244/2503] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:21.537094 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.537414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.537723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.555434 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:21.556668 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.564370 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1260/2504] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:21.568977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.578560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2b87-917e-4482-b884-e1c21aab20f4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.583757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.584444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.584713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.584995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.585758 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:21.586080 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.587690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.599262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1245/2505] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:21.603608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.612906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098905bd-2219-4c79-a3cf-80a65bde91f7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.617483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.618172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.618462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.618738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.619425 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.621325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.647411 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1261/2506] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:21.651779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.662295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24342b5-11a5-49f3-b5a4-442a01b377cc admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.718154 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.718886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.719124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.719408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.720100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.722838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.731940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:21.745357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1246/2507] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:21.749404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.759813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0247d3-9d88-4a4f-a964-313793dcbd05 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.764433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.765097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.765392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.765753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.766489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.768124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.779216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1262/2508] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:21.783143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.794151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c3282-5d8f-498f-ae2f-7f2df5b3ab2b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.798791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15cf4b-6f1f-486f-94ba-a3bb8ec4f175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.799450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15cf4b-6f1f-486f-94ba-a3bb8ec4f175 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.800257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15cf4b-6f1f-486f-94ba-a3bb8ec4f175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.800545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15cf4b-6f1f-486f-94ba-a3bb8ec4f175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.801277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15cf4b-6f1f-486f-94ba-a3bb8ec4f175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.802939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15cf4b-6f1f-486f-94ba-a3bb8ec4f175 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.812685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1247/2509] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:21.816773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:21.828145 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7446edc-b861-4cd3-a321-1c9803d6dca9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:21.833449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:21.834373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.834661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.834939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] PATH_INFO: `/v3/projects/46ff4d24778347d98b6981085b924f0d/users/55d2bc8a992842268d1d8ad03d31c3e4/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.849818 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ff4d24778347d98b6981085b924f0d, user_id=55d2bc8a992842268d1d8ad03d31c3e4, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:21.852735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:21.866034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1263/2510] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:21 2022] PUT /identity/v3/projects/46ff4d24778347d98b6981085b924f0d/users/55d2bc8a992842268d1d8ad03d31c3e4/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:21.877986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:21.878774 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:21.879137 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:21.904976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] MFA Rules not processed for user `55d2bc8a992842268d1d8ad03d31c3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:21.906198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:21.971357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1248/2511] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:21.976686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.040392 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88c739-39e1-4c6d-bf28-60c2528c3c24 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.045032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.045704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.045995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.046270 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.046969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.048847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.074146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1264/2512] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:21 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:22.078479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.089373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f7d8-c65d-45c6-8154-0f5610cfa99d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.146321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.147006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.147294 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.147579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.148284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.150222 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.170603 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1249/2513] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:22 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:22.174594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.185728 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd863308-6757-4d91-af8c-c87e838b7b49 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.190699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.191452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.191732 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.192016 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.192754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.194400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.205203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1265/2514] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:22.209462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.219407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4461e-18c3-4d5b-aeb1-344e3703e8d7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.224731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.225555 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.225881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.226187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] PATH_INFO: `/v3/projects/a4ebc8c8b1b445999966a8cf5fdc6ae1/users/91f623c78e86415db0159c700e4e40a6/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.241547 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ebc8c8b1b445999966a8cf5fdc6ae1, user_id=91f623c78e86415db0159c700e4e40a6, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.244238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.257825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1250/2515] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:22 2022] PUT /identity/v3/projects/a4ebc8c8b1b445999966a8cf5fdc6ae1/users/91f623c78e86415db0159c700e4e40a6/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:22.261788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.272362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a89e72-4de1-4a1f-b10d-c7810f8f7e79 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.305024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:22.333882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb191b-81d3-47b7-97c1-37575cfb4a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.334561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb191b-81d3-47b7-97c1-37575cfb4a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.334841 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb191b-81d3-47b7-97c1-37575cfb4a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.335115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb191b-81d3-47b7-97c1-37575cfb4a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.335834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb191b-81d3-47b7-97c1-37575cfb4a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.337417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb191b-81d3-47b7-97c1-37575cfb4a2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.347701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1266/2516] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:22.352308 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.362698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4e6d0-3c1d-48ff-b0c3-c4e58504615b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.367340 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.368225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.368511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.368793 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] PATH_INFO: `/v3/projects/a4ebc8c8b1b445999966a8cf5fdc6ae1/users/91f623c78e86415db0159c700e4e40a6/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.370789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ebc8c8b1b445999966a8cf5fdc6ae1, user_id=91f623c78e86415db0159c700e4e40a6, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.373443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.384348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1251/2517] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:22 2022] PUT /identity/v3/projects/a4ebc8c8b1b445999966a8cf5fdc6ae1/users/91f623c78e86415db0159c700e4e40a6/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:22.395661 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.395957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.396241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.422216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] MFA Rules not processed for user `91f623c78e86415db0159c700e4e40a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:22.423417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:22.494369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1267/2518] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:22.506263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.508047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:22.537136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:22.567594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8eec3-42fb-4cd0-b52b-97214fcf10b4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.571236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.571882 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.572171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.572455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.581975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.583390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.592631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1252/2519] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:22.670278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:22.680216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33dbd4-aadf-4970-a792-97e7363ffc3b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:22.683772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad2b9f-8868-434e-ad62-aa0db1aaa176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:22.684400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad2b9f-8868-434e-ad62-aa0db1aaa176 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:22.684685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad2b9f-8868-434e-ad62-aa0db1aaa176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:22.684962 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad2b9f-8868-434e-ad62-aa0db1aaa176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:22.741349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad2b9f-8868-434e-ad62-aa0db1aaa176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:22.742823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad2b9f-8868-434e-ad62-aa0db1aaa176 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:22.751568 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1268/2520] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:27.431470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:27.495410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024efc83-ce7c-4754-be9f-5dae69641f85 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:27.499877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:27.500737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.501021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.501313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] PATH_INFO: `/v3/users/55d2bc8a992842268d1d8ad03d31c3e4` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.502495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d2bc8a992842268d1d8ad03d31c3e4)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:27.504401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:27.549572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d2bc8a992842268d1d8ad03d31c3e4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.561148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d2bc8a992842268d1d8ad03d31c3e4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.568590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d2bc8a992842268d1d8ad03d31c3e4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.575515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d2bc8a992842268d1d8ad03d31c3e4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.576181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d2bc8a992842268d1d8ad03d31c3e4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:27.582104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1253/2521] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:27 2022] DELETE /identity/v3/users/55d2bc8a992842268d1d8ad03d31c3e4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:27.587238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.587744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.587854 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.604491 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:27.605714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:27.652394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:27.664116 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1269/2522] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:27.673769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:27.675220 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:27.728244 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:27.734587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c5d22-20f3-4f6b-a762-51d2664e9723 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:27.739563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:27.740434 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:27.740723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:27.741001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:27.750058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:27.751474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:27.761317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1254/2523] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:28.034021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.035613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.047975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f59eb-5eba-4051-98aa-bf52819a8bb5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.053290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.054189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.054486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.054770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] PATH_INFO: `/v3/projects/46ff4d24778347d98b6981085b924f0d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.055982 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ff4d24778347d98b6981085b924f0d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.058458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.105825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ff4d24778347d98b6981085b924f0d', 'request_id': 'req-62fa180b-6cef-4c77-b473-22b0fd7d94bb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.113064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ff4d24778347d98b6981085b924f0d', 'request_id': 'req-62fa180b-6cef-4c77-b473-22b0fd7d94bb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.119480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ff4d24778347d98b6981085b924f0d', 'request_id': 'req-62fa180b-6cef-4c77-b473-22b0fd7d94bb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.130663 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] The token cache is being invalidate because project 46ff4d24778347d98b6981085b924f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:28.130957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.131592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.133213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1270/2524] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:28 2022] DELETE /identity/v3/projects/46ff4d24778347d98b6981085b924f0d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:28.137189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.146800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0be86-6084-43d2-ab01-82aa69ed6046 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.202519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.203395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.203686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.203968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] PATH_INFO: `/v3/users/91f623c78e86415db0159c700e4e40a6` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.205159 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f623c78e86415db0159c700e4e40a6)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.207103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.253984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f623c78e86415db0159c700e4e40a6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.270423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f623c78e86415db0159c700e4e40a6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.279391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f623c78e86415db0159c700e4e40a6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.288179 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f623c78e86415db0159c700e4e40a6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.289056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-40c3a4fa-6d89-42e0-a00a-987bc200782a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f623c78e86415db0159c700e4e40a6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.296005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1255/2525] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:28 2022] DELETE /identity/v3/users/91f623c78e86415db0159c700e4e40a6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:28.573179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.574707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.589879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa180b-6cef-4c77-b473-22b0fd7d94bb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.650301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.651083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.651369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.651646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] PATH_INFO: `/v3/projects/a4ebc8c8b1b445999966a8cf5fdc6ae1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.652831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ebc8c8b1b445999966a8cf5fdc6ae1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.654778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.700957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ebc8c8b1b445999966a8cf5fdc6ae1', 'request_id': 'req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.708176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ebc8c8b1b445999966a8cf5fdc6ae1', 'request_id': 'req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.713958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ebc8c8b1b445999966a8cf5fdc6ae1', 'request_id': 'req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.724838 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] The token cache is being invalidate because project a4ebc8c8b1b445999966a8cf5fdc6ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:28.725129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.725810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-413b1418-fc9c-4acf-b94a-d3bd8e637f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:28.727606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1271/2526] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:28 2022] DELETE /identity/v3/projects/a4ebc8c8b1b445999966a8cf5fdc6ae1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:28.733224 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.733730 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.733839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.751519 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:28.752717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.811263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1256/2527] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:28.819745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.820039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.820320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.838250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:28.839469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:28.847290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1272/2528] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:28.852245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.862024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a0f05-dcda-40a7-8a09-a253cb8ca44c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.867305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.867967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.868259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.868544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.869285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:28.869602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.871247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.873602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:28.885116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1257/2529] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:28.889450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.893945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:28.900043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d334be2-b725-49bf-a159-9d049b5623a2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:28.905652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:28.906303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:28.906592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:28.906873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:28.907567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:28.909637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:28.935363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1273/2530] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:28 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:28.939561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:28.949517 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f1a4b-7162-4f1b-b98b-0feae2634aff admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.005163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.005866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.006152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.006437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.007145 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.009053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.029500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1258/2531] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:28 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:29.033213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.043958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270c4a4-0dc9-4467-896c-c91c98fb4e02 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.048552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.049216 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.049511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.049835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.050571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.052832 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.063192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1274/2532] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:29.067008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.077314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5632db4-a0b9-462a-a3a4-ba8337beeca0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.081840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66235362-b9b0-4481-b73e-62f05ec4ed3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.082499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66235362-b9b0-4481-b73e-62f05ec4ed3c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.082818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66235362-b9b0-4481-b73e-62f05ec4ed3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.083349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66235362-b9b0-4481-b73e-62f05ec4ed3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.084261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66235362-b9b0-4481-b73e-62f05ec4ed3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.085916 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66235362-b9b0-4481-b73e-62f05ec4ed3c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.095897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1259/2533] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:29.100556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.110983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6a12-5510-424c-9d2a-9ae33055e377 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.116371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.117281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.117572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.117909 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] PATH_INFO: `/v3/projects/db880be608264460afa8d78e7ec25b96/users/2040dd75ad074337ada21e2d4570662d/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.133560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db880be608264460afa8d78e7ec25b96, user_id=2040dd75ad074337ada21e2d4570662d, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.136229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.148915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1275/2534] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:29 2022] PUT /identity/v3/projects/db880be608264460afa8d78e7ec25b96/users/2040dd75ad074337ada21e2d4570662d/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:29.160315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.160556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.161360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.187941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] MFA Rules not processed for user `2040dd75ad074337ada21e2d4570662d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.189195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.253635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1260/2535] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.265052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.266744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.324168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c7e6-1ec9-4f3e-aca9-474d899b55f6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.328626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfde3d2-47bd-4a3f-ba0e-a71bea0ae0b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.329271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfde3d2-47bd-4a3f-ba0e-a71bea0ae0b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.329553 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfde3d2-47bd-4a3f-ba0e-a71bea0ae0b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.329864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfde3d2-47bd-4a3f-ba0e-a71bea0ae0b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.339397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfde3d2-47bd-4a3f-ba0e-a71bea0ae0b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.340811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfde3d2-47bd-4a3f-ba0e-a71bea0ae0b3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.350792 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1276/2536] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:29.370481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.433120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e3f5b-f567-4283-a759-d0f711ac0963 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.437757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.438690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.438979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.439284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] PATH_INFO: `/v3/users/2040dd75ad074337ada21e2d4570662d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.440499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2040dd75ad074337ada21e2d4570662d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.442490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.447091 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:29.493153 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2040dd75ad074337ada21e2d4570662d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.504889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2040dd75ad074337ada21e2d4570662d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.511684 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2040dd75ad074337ada21e2d4570662d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.519890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2040dd75ad074337ada21e2d4570662d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.521374 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2040dd75ad074337ada21e2d4570662d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:29.527935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1261/2537] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:29 2022] DELETE /identity/v3/users/2040dd75ad074337ada21e2d4570662d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:51:29.532570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.532868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.533150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.550747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:29.552033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.609674 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1277/2538] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:29.619794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.621344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.681437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e0e55-0428-4170-b6fa-0bcd408dca6e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:29.685163 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8f669-3366-494b-8256-caa8c62c07bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:29.685854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8f669-3366-494b-8256-caa8c62c07bc service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:29.686143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8f669-3366-494b-8256-caa8c62c07bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:29.686422 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8f669-3366-494b-8256-caa8c62c07bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:29.695496 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8f669-3366-494b-8256-caa8c62c07bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:29.696905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8f669-3366-494b-8256-caa8c62c07bc service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:29.706254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1262/2539] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:29.979421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:29.980946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:29.993849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fd28-cbc2-4f26-80d9-0f4d3c555e93 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.000480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.002056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.002565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.003173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] PATH_INFO: `/v3/projects/db880be608264460afa8d78e7ec25b96` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.005358 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] RBAC: Authorizing `identity:delete_project(project_id=db880be608264460afa8d78e7ec25b96)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.009052 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.052993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:30.056920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db880be608264460afa8d78e7ec25b96', 'request_id': 'req-31047af8-d50a-495e-81e3-abdd077fefae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.065242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db880be608264460afa8d78e7ec25b96', 'request_id': 'req-31047af8-d50a-495e-81e3-abdd077fefae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.070796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db880be608264460afa8d78e7ec25b96', 'request_id': 'req-31047af8-d50a-495e-81e3-abdd077fefae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.081466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] The token cache is being invalidate because project db880be608264460afa8d78e7ec25b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:30.081796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.082446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31047af8-d50a-495e-81e3-abdd077fefae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:30.084517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1278/2540] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:29 2022] DELETE /identity/v3/projects/db880be608264460afa8d78e7ec25b96 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:30.089723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.090185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.090283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.108658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:30.109886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.168284 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1263/2541] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:30.175916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.176217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.176989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.195567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:30.196780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.204676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1279/2542] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:30.208889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.218736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a304c-ccc1-46f7-8e8c-e61376c3ac4b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.223567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.224171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.224456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.224738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.225467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:30.225814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.227664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.239609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1264/2543] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:51:30.244218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.253455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19717d3-b0e1-4246-abf6-c8f20c07c4f2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.259125 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.259780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.260073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.260359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.261060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.263000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.288983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1280/2544] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:30.293198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.303984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42701687-81c1-4d17-bec9-44b54ae6c5e2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.312692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:30.363475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.364236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.364529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.364816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.365528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.367514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.388053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1265/2545] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:30.392140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.403083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838baa5d-4492-45bc-bfa7-bd5a0ef0a57e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.407698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.408396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.408686 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.408969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.409733 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.411356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.421627 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1281/2546] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:30.425443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.435357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec490b9d-6eca-4395-b21f-206531ecb493 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.440133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20d838-9cd4-4dbd-90f2-5fd5bb490bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.440790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20d838-9cd4-4dbd-90f2-5fd5bb490bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.441085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20d838-9cd4-4dbd-90f2-5fd5bb490bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.441368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20d838-9cd4-4dbd-90f2-5fd5bb490bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.442659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20d838-9cd4-4dbd-90f2-5fd5bb490bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.444291 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20d838-9cd4-4dbd-90f2-5fd5bb490bb9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.459275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1266/2547] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:30.463462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.473419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac31960-487c-4a59-9eb6-d55fe0967041 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.478529 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.479351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.479643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.479928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] PATH_INFO: `/v3/projects/8d40b771f30d4e668e0c908fbd0da517/users/44f839b774b249dd9d7d3061feb2cd41/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.494115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d40b771f30d4e668e0c908fbd0da517, user_id=44f839b774b249dd9d7d3061feb2cd41, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.496765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.510413 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1282/2548] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:30 2022] PUT /identity/v3/projects/8d40b771f30d4e668e0c908fbd0da517/users/44f839b774b249dd9d7d3061feb2cd41/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:30.515473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a869-48af-41eb-a858-5825b573ef99 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.515753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a869-48af-41eb-a858-5825b573ef99 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.516039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a869-48af-41eb-a858-5825b573ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.534286 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d893a869-48af-41eb-a858-5825b573ef99 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:30.535491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d893a869-48af-41eb-a858-5825b573ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.593375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1267/2549] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:30.603817 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:30.605506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:30.662379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bf37a-aec0-45a6-834c-0bd051428844 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:30.666051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:30.666698 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:30.666986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:30.667267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:30.676198 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:30.678563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:30.688431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1283/2550] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:31.272843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83975fa9-83e9-4a49-bda3-a425767056eb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.273368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83975fa9-83e9-4a49-bda3-a425767056eb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.273466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83975fa9-83e9-4a49-bda3-a425767056eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.300438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-83975fa9-83e9-4a49-bda3-a425767056eb None None] MFA Rules not processed for user `44f839b774b249dd9d7d3061feb2cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:31.302128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-83975fa9-83e9-4a49-bda3-a425767056eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.370178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1268/2551] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:51:31.382118 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:31.383676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:31.443943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190821b2-c06a-44d3-bf8e-bb820431a963 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:31.447905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:31.448585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:31.448876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:31.449160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:31.460480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:31.461979 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:31.472188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1284/2552] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:32.826272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee840d-29d8-4130-9646-be299aceb962 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:32.826808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee840d-29d8-4130-9646-be299aceb962 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:32.826808 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee840d-29d8-4130-9646-be299aceb962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:32.844649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-32ee840d-29d8-4130-9646-be299aceb962 None None] MFA Rules not processed for user `44f839b774b249dd9d7d3061feb2cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:32.845885 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-32ee840d-29d8-4130-9646-be299aceb962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:32.853647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1269/2553] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:33.862366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:33.863801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:33.879026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:33.936160 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbb0a-720e-4b44-a965-8020d802aa59 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:33.941106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:33.941789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:33.942076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:33.943006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:33.945020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:33.956237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:33.957880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac88db9-2d41-4026-815a-5509686aa31b service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:33.967801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1285/2554] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:42.839433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:43.963476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=26 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:48.262360 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:48.262866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:48.262972 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:48.280392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] MFA Rules not processed for user `44f839b774b249dd9d7d3061feb2cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:48.281607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:48.289127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1270/2555] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:58.134700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:58.135265 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:58.135366 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:58.152274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] MFA Rules not processed for user `6caa64aecb914e968d057b6b593f667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:58.153479 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:58.216018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1286/2556] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:58.275212 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:59.956959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:59.958368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:59.970395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400805b-6920-4a1f-99ce-45ac2da7e069 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:59.974287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:59.974941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:59.975228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:59.975509 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:59.977252 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:59.989738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:59.991655 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:00.001599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1271/2557] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:04.827740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:04.829246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:04.851734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9b32a-434e-4b07-b5b9-1cbf451e359c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:04.857327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:04.858254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:04.858549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:04.858837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] PATH_INFO: `/v3/users/1d58ac7f49454fd6a5b5eef6a1e8b2d8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:04.860050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d58ac7f49454fd6a5b5eef6a1e8b2d8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:04.862066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:04.911080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d58ac7f49454fd6a5b5eef6a1e8b2d8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:04.911754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d58ac7f49454fd6a5b5eef6a1e8b2d8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:04.918665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d58ac7f49454fd6a5b5eef6a1e8b2d8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:04.919297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d58ac7f49454fd6a5b5eef6a1e8b2d8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:04.926689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-dac3b637-456c-4663-80ef-4bd6c33ef923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d58ac7f49454fd6a5b5eef6a1e8b2d8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:04.943866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1287/2558] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:04 2022] DELETE /identity/v3/users/1d58ac7f49454fd6a5b5eef6a1e8b2d8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:05.121616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.132260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ba8e9-d7b2-4775-8765-515864f980d0 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.190072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.190942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.191231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.191514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] PATH_INFO: `/v3/projects/4bf1cdbd5dc9471f9f1732600ab98c24` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.192690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf1cdbd5dc9471f9f1732600ab98c24)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.195454 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.240845 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf1cdbd5dc9471f9f1732600ab98c24', 'request_id': 'req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.246962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf1cdbd5dc9471f9f1732600ab98c24', 'request_id': 'req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.252958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf1cdbd5dc9471f9f1732600ab98c24', 'request_id': 'req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.265979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] The token cache is being invalidate because project 4bf1cdbd5dc9471f9f1732600ab98c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:05.266275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.266898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f6c4ddf4-5bed-4e69-8953-992ab1126cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:05.268472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1272/2559] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:05 2022] DELETE /identity/v3/projects/4bf1cdbd5dc9471f9f1732600ab98c24 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:05.273484 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.273863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.274155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.293692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:05.294920 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:05.355588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1288/2560] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:05.364436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.364744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.365033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.384864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:05.386124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:05.394209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1273/2561] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:05.398775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.408946 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34fbd1-e995-46bd-b6e7-fba20096f1b8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.414078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.414788 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.415060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.415342 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.416089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:05.416401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.418044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.432871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1289/2562] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:05.437236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.447512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678568e5-dfb6-4fa0-b2a1-d930d2ce814f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.452290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.453181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.453503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.453832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.454570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.456515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.482877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1274/2563] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:05.487876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.495471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:05.499252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b9550-526a-4fe1-9728-2687f8cb99fc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.567255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.567936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.568225 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.568506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.569761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.571699 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.593465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1290/2564] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:05.597721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.610953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d809e20-8a95-4afc-bec2-d7dca85e5e3d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.617059 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.618032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.618344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.618614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.619355 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.620995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.633498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1275/2565] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:05.637528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.647924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187a3a-6287-4930-89f3-f19b4bbfb385 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.652693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e75d2f-68f4-4c84-a659-498ba4e84573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.653363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e75d2f-68f4-4c84-a659-498ba4e84573 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.653676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e75d2f-68f4-4c84-a659-498ba4e84573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.653972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e75d2f-68f4-4c84-a659-498ba4e84573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.654709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e75d2f-68f4-4c84-a659-498ba4e84573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.656312 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e75d2f-68f4-4c84-a659-498ba4e84573 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.667508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1291/2566] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:05.671970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.682812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e8d70-86ea-4187-b71c-63ba74cc832b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.687556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.688444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.688737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.689022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] PATH_INFO: `/v3/projects/acd4e096a0ce439ea510ea4ccc880b81/users/ae08d25a52534854844edf8ac57bf7dc/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.704203 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd4e096a0ce439ea510ea4ccc880b81, user_id=ae08d25a52534854844edf8ac57bf7dc, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.706812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.720191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1276/2567] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:05 2022] PUT /identity/v3/projects/acd4e096a0ce439ea510ea4ccc880b81/users/ae08d25a52534854844edf8ac57bf7dc/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:05.730763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.731077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.731445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.760329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] MFA Rules not processed for user `ae08d25a52534854844edf8ac57bf7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:05.761552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:05.828170 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1292/2568] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:05.832854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.848843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:05.910862 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943f19-0f21-4213-a1fc-6562c766f8e6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:05.916472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:05.917208 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:05.917588 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:05.917877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:05.918618 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:05.920572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:05.957768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1277/2569] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:05.959761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:05.971211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ecf91-9672-403f-9f40-f5f5c6830ebb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.033429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.034156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.034443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.034726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.035439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.037582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.061722 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1293/2570] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:05 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:06.062941 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.074363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79726be4-a7a0-4bee-95ad-160fb4ba6617 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.080365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.081115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.081472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.081855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.082665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.084358 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.100641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1278/2571] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:06.102063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.107397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:06.114352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861272e3-4c43-4947-ae0a-20c705d27aac admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.121989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.122945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.123235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.123515 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] PATH_INFO: `/v3/projects/7ecc4e48ec2d492c9f255ed9ff73f6a0/users/ef8b99edd79f4cb0ae138c266668b44a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.139503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecc4e48ec2d492c9f255ed9ff73f6a0, user_id=ef8b99edd79f4cb0ae138c266668b44a, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.142203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.156875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1294/2572] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:06 2022] PUT /identity/v3/projects/7ecc4e48ec2d492c9f255ed9ff73f6a0/users/ef8b99edd79f4cb0ae138c266668b44a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:06.160837 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.171435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbda59-66db-4ff3-9272-82489c652194 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.227860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c057a8-48b9-425e-8611-74e9bee62bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.228731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c057a8-48b9-425e-8611-74e9bee62bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.229023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c057a8-48b9-425e-8611-74e9bee62bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.229306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c057a8-48b9-425e-8611-74e9bee62bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.230112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c057a8-48b9-425e-8611-74e9bee62bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.231745 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c057a8-48b9-425e-8611-74e9bee62bf0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.242500 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1279/2573] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:06.247148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.258639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa3fa3-c1d1-43aa-85c3-78460240bf70 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.263303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.264183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.264500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.264786 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] PATH_INFO: `/v3/projects/7ecc4e48ec2d492c9f255ed9ff73f6a0/users/ef8b99edd79f4cb0ae138c266668b44a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.266777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecc4e48ec2d492c9f255ed9ff73f6a0, user_id=ef8b99edd79f4cb0ae138c266668b44a, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.269412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.279940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1295/2574] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:06 2022] PUT /identity/v3/projects/7ecc4e48ec2d492c9f255ed9ff73f6a0/users/ef8b99edd79f4cb0ae138c266668b44a/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:06.290494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.290791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.291078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.318231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] MFA Rules not processed for user `ef8b99edd79f4cb0ae138c266668b44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:06.319504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:06.388381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1280/2575] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:06.399959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.401694 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:06.460074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450bc98-6077-4115-bff7-942afd0c0eb3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.463858 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50adb9-ec58-4033-bd78-ae30b7a4b87e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.464535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50adb9-ec58-4033-bd78-ae30b7a4b87e service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.464849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50adb9-ec58-4033-bd78-ae30b7a4b87e service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.465130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50adb9-ec58-4033-bd78-ae30b7a4b87e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.474049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50adb9-ec58-4033-bd78-ae30b7a4b87e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.476010 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50adb9-ec58-4033-bd78-ae30b7a4b87e service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:06.486276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1296/2576] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:06.888322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:06.897898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:06.952433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca6be0-e248-46d2-a891-8933ca182d6c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:06.957448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:06.958433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:06.958670 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:06.958951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] PATH_INFO: `/v3/users/ae08d25a52534854844edf8ac57bf7dc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:06.960738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae08d25a52534854844edf8ac57bf7dc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:06.962567 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.042177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae08d25a52534854844edf8ac57bf7dc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.056648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae08d25a52534854844edf8ac57bf7dc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.064766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae08d25a52534854844edf8ac57bf7dc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.076409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae08d25a52534854844edf8ac57bf7dc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.077828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae08d25a52534854844edf8ac57bf7dc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.086139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1281/2577] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:06 2022] DELETE /identity/v3/users/ae08d25a52534854844edf8ac57bf7dc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:07.091071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.091509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.091900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.132630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:07.134454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:07.215278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1297/2578] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:07.225051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.226514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:07.289243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57a8c5-ab3f-44ba-a800-e195607358ad admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.293106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.293831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.294177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.295105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.306144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.307998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.320946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1282/2579] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:07.645894 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.647866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:07.662097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bbcf1-143b-45bb-abb0-ec3d1adf8dd1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.667206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.668153 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.668618 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.668903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] PATH_INFO: `/v3/projects/acd4e096a0ce439ea510ea4ccc880b81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.670165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd4e096a0ce439ea510ea4ccc880b81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.672093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.723228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd4e096a0ce439ea510ea4ccc880b81', 'request_id': 'req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.731552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd4e096a0ce439ea510ea4ccc880b81', 'request_id': 'req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.736750 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:07.738139 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd4e096a0ce439ea510ea4ccc880b81', 'request_id': 'req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.753227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] The token cache is being invalidate because project acd4e096a0ce439ea510ea4ccc880b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:07.753450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.754107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.755830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1298/2580] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:07 2022] DELETE /identity/v3/projects/acd4e096a0ce439ea510ea4ccc880b81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:07.760689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:07.771622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6d90-2413-4666-911b-2fc8aaf016d9 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:07.803160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:07.834563 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:07.835481 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:07.835769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:07.836056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] PATH_INFO: `/v3/users/ef8b99edd79f4cb0ae138c266668b44a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:07.837259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8b99edd79f4cb0ae138c266668b44a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:07.839251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:07.886557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8b99edd79f4cb0ae138c266668b44a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.899676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8b99edd79f4cb0ae138c266668b44a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.907126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8b99edd79f4cb0ae138c266668b44a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.916390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8b99edd79f4cb0ae138c266668b44a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.917105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e56b88b3-f2a4-4d8f-82b1-1f61f1239591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8b99edd79f4cb0ae138c266668b44a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:07.924081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1283/2581] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:07 2022] DELETE /identity/v3/users/ef8b99edd79f4cb0ae138c266668b44a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:08.197949 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.199497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:08.213997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a1c23-8efb-4cb2-bc22-053dad9b2ef0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.275427 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.276299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.276583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.276865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] PATH_INFO: `/v3/projects/7ecc4e48ec2d492c9f255ed9ff73f6a0` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.278080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ecc4e48ec2d492c9f255ed9ff73f6a0)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.280001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.331174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ecc4e48ec2d492c9f255ed9ff73f6a0', 'request_id': 'req-944e9205-f012-46f1-ba9f-219f66e3c861'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:08.340974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ecc4e48ec2d492c9f255ed9ff73f6a0', 'request_id': 'req-944e9205-f012-46f1-ba9f-219f66e3c861'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:08.355066 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ecc4e48ec2d492c9f255ed9ff73f6a0', 'request_id': 'req-944e9205-f012-46f1-ba9f-219f66e3c861'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:08.366269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] The token cache is being invalidate because project 7ecc4e48ec2d492c9f255ed9ff73f6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:08.366562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:08.367373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-944e9205-f012-46f1-ba9f-219f66e3c861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:08.368941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1299/2582] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:08 2022] DELETE /identity/v3/projects/7ecc4e48ec2d492c9f255ed9ff73f6a0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:08.373977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.374443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.374579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.393644 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:08.395004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:08.457790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1284/2583] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:08.467740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.468080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.468381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.488362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:08.489565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:08.497716 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1300/2584] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:08.501964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.511634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c328bd6-0931-45ae-81a0-69cfdd7e3ce3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.516186 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.516844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.517132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.517417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.518185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:08.518499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.520087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.531915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1285/2585] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:08.536193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.546362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671798b-d45f-41b9-b0d0-e49663dff89e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.550908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.551576 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.551846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.552129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.552820 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.555309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.580205 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1301/2586] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:08.584332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.595341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22e8fc-5fd8-40fa-8174-2140f1511dda admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.627764 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:08.654974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.655681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.655970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.656253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.656947 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.658890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.678411 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1286/2587] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:08.682634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.692876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f04d-73fe-46d5-933b-0abf91887bd6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.698135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.698801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.699086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.699365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.700091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.701707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.712673 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1302/2588] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:08.716443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.726574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699f5b8-a375-4400-a7fd-12b48cd8d405 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.731742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f6436-a384-4e12-ae0b-2260b61b1e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.732418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6436-a384-4e12-ae0b-2260b61b1e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.732763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6436-a384-4e12-ae0b-2260b61b1e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.733074 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6436-a384-4e12-ae0b-2260b61b1e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.734047 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f6436-a384-4e12-ae0b-2260b61b1e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.735690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f6436-a384-4e12-ae0b-2260b61b1e43 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.746041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1287/2589] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:08.750209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.760644 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b5f11-6a1f-49d1-bbf2-e567142cc462 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.765432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.766349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.766641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.766925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] PATH_INFO: `/v3/projects/a23d73d9ca5a4a8ba15eef17c2d35493/users/d7f11a8453cf46428ab1dd0fdd533190/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.781908 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23d73d9ca5a4a8ba15eef17c2d35493, user_id=d7f11a8453cf46428ab1dd0fdd533190, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.784614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:08.797801 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1303/2590] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:08 2022] PUT /identity/v3/projects/a23d73d9ca5a4a8ba15eef17c2d35493/users/d7f11a8453cf46428ab1dd0fdd533190/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:08.808615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.808903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.809193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.836705 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] MFA Rules not processed for user `d7f11a8453cf46428ab1dd0fdd533190`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:08.838029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:08.904472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1288/2591] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:08.908987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:08.928764 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:08.971593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b89fc-a650-4f09-8bc1-9ee2f7a817de admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:08.977311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:08.977996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:08.978541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:08.978829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:08.979535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:08.981463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.005397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1304/2592] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:08 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:09.009718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.020653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5433dc-2cad-4a06-9db5-a03d09e9b044 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.078257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.078858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.079152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.079438 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.080151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.082108 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.104846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1289/2593] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:09 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:09.109055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.120167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a11c0-a184-4818-a2fc-1a4708bca316 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.124828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.125487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.125810 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.126109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.126821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.128416 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.139513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1305/2594] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:09.143599 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.154785 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30752-4374-409b-a0f9-da9637b64091 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.160146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.161037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.161327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.161611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] PATH_INFO: `/v3/projects/2da2e6c3a7d0474bb39eecd2fd07e57b/users/a3ce2e09bd624579b5e6e0a0da824445/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.176887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da2e6c3a7d0474bb39eecd2fd07e57b, user_id=a3ce2e09bd624579b5e6e0a0da824445, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.179665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.192991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1290/2595] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:09 2022] PUT /identity/v3/projects/2da2e6c3a7d0474bb39eecd2fd07e57b/users/a3ce2e09bd624579b5e6e0a0da824445/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:09.196796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.208609 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf44f8-6d28-474f-8023-2996033a68f7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.265555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221f32a-50cc-4cb3-82ee-0a43e74ced64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.266274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221f32a-50cc-4cb3-82ee-0a43e74ced64 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.266565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221f32a-50cc-4cb3-82ee-0a43e74ced64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.266849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221f32a-50cc-4cb3-82ee-0a43e74ced64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.267586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221f32a-50cc-4cb3-82ee-0a43e74ced64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.269227 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221f32a-50cc-4cb3-82ee-0a43e74ced64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.279555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1306/2596] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:09.283690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.295520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28962779-b130-4b42-af62-7d381157d0d0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.300354 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.301245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.301538 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.301858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] PATH_INFO: `/v3/projects/2da2e6c3a7d0474bb39eecd2fd07e57b/users/a3ce2e09bd624579b5e6e0a0da824445/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.303876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da2e6c3a7d0474bb39eecd2fd07e57b, user_id=a3ce2e09bd624579b5e6e0a0da824445, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.306772 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.312669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:09.318846 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1291/2597] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:09 2022] PUT /identity/v3/projects/2da2e6c3a7d0474bb39eecd2fd07e57b/users/a3ce2e09bd624579b5e6e0a0da824445/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:09.332490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.332747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.333019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.363378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] MFA Rules not processed for user `a3ce2e09bd624579b5e6e0a0da824445`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:09.364621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.434327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1307/2598] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:09.445790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.448194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.508125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b73958-27f9-4e7f-8be1-9f6a93f9a7e9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.511866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be124e-fa48-4a7b-9903-02e66b58a568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.512524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be124e-fa48-4a7b-9903-02e66b58a568 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.512816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be124e-fa48-4a7b-9903-02e66b58a568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.513102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be124e-fa48-4a7b-9903-02e66b58a568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.523259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be124e-fa48-4a7b-9903-02e66b58a568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.524691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be124e-fa48-4a7b-9903-02e66b58a568 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.534371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1292/2599] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:09.561202 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.574977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:09.624649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66214c-6eca-4b5e-ba76-0cf7fed39698 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.629211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.630113 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.630404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.630687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] PATH_INFO: `/v3/users/d7f11a8453cf46428ab1dd0fdd533190` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.632430 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f11a8453cf46428ab1dd0fdd533190)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.634394 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.682432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f11a8453cf46428ab1dd0fdd533190'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.683253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f11a8453cf46428ab1dd0fdd533190'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.690407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f11a8453cf46428ab1dd0fdd533190'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.691080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f11a8453cf46428ab1dd0fdd533190'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.699217 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f11a8453cf46428ab1dd0fdd533190'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:09.716782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1308/2600] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:09 2022] DELETE /identity/v3/users/d7f11a8453cf46428ab1dd0fdd533190 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:09.721317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.721616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.721942 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.739524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:09.740735 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.798902 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1293/2601] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:09.809023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:09.810473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:09.868278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e426f-a325-492b-9517-90709a1f4505 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:09.871864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:09.872518 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:09.872807 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:09.873092 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:09.883009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:09.884373 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:09.893915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1309/2602] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:10.160593 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:10.162112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.174712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020534e-3966-4286-bef1-a8b2f40dc0a8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:10.180253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:10.181142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.181430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.181747 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] PATH_INFO: `/v3/projects/a23d73d9ca5a4a8ba15eef17c2d35493` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.182982 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23d73d9ca5a4a8ba15eef17c2d35493)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:10.184896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.239475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23d73d9ca5a4a8ba15eef17c2d35493', 'request_id': 'req-e2309877-b15c-4c82-954a-0c059b377185'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.245087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23d73d9ca5a4a8ba15eef17c2d35493', 'request_id': 'req-e2309877-b15c-4c82-954a-0c059b377185'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.251251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23d73d9ca5a4a8ba15eef17c2d35493', 'request_id': 'req-e2309877-b15c-4c82-954a-0c059b377185'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.264132 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] The token cache is being invalidate because project a23d73d9ca5a4a8ba15eef17c2d35493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:10.264426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.265068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.266649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1294/2603] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:10 2022] DELETE /identity/v3/projects/a23d73d9ca5a4a8ba15eef17c2d35493 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:10.270545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:10.280742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81981457-38c4-4e39-a0e4-7614b6bc945b service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:10.314088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:10.342281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:10.343171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.343463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.343747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] PATH_INFO: `/v3/users/a3ce2e09bd624579b5e6e0a0da824445` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.344954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ce2e09bd624579b5e6e0a0da824445)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:10.346907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.395239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ce2e09bd624579b5e6e0a0da824445'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.395896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ce2e09bd624579b5e6e0a0da824445'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.403407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ce2e09bd624579b5e6e0a0da824445'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.403994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ce2e09bd624579b5e6e0a0da824445'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.411176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-796f7afa-a221-4402-bc2e-0c45c7d182f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ce2e09bd624579b5e6e0a0da824445'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.428222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1310/2604] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:10 2022] DELETE /identity/v3/users/a3ce2e09bd624579b5e6e0a0da824445 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:10.713933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:10.715457 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.731175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309877-b15c-4c82-954a-0c059b377185 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:10.756782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:10.796007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:10.796879 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.797169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.797452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] PATH_INFO: `/v3/projects/2da2e6c3a7d0474bb39eecd2fd07e57b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.798679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da2e6c3a7d0474bb39eecd2fd07e57b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:10.800569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:10.846260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da2e6c3a7d0474bb39eecd2fd07e57b', 'request_id': 'req-3e04b917-ed69-4581-87db-f0050b960830'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.852292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da2e6c3a7d0474bb39eecd2fd07e57b', 'request_id': 'req-3e04b917-ed69-4581-87db-f0050b960830'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.858692 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da2e6c3a7d0474bb39eecd2fd07e57b', 'request_id': 'req-3e04b917-ed69-4581-87db-f0050b960830'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.874984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] The token cache is being invalidate because project 2da2e6c3a7d0474bb39eecd2fd07e57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:10.875307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.875935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e04b917-ed69-4581-87db-f0050b960830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:10.877489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1295/2605] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:10 2022] DELETE /identity/v3/projects/2da2e6c3a7d0474bb39eecd2fd07e57b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:10.882377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.882672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.882953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.900072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:10.901269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.959881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1311/2606] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:10.968082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:10.968381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:10.968669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:10.988766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:10.990018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:10.998611 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1296/2607] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:11.002727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.012740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694c18d-5c29-4c13-b8ed-1cedd224e41d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.017372 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.018060 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.018350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.018635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.019380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:11.019696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.021282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.034879 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1312/2608] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:11.040925 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.050825 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e09ba-b980-4840-997e-1754e202bd6f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.056671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.057336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.057622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.057937 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.058629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.060505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.092450 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1297/2609] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:11.097827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.112958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3410c32-002e-4af5-a3f9-8a23dba1d5b4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.154528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:11.173542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.174261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.174575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.174860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.175551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.177545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.197865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1313/2610] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:11.201835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.212690 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ac8aa-2fd5-4393-932b-8b46be45226d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.218225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.218893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.219175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.219459 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.220185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.221905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.233429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1298/2611] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:11.237714 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.247978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fe9a6-0a5c-40b3-93c8-04550cf33d7d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.252738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.253405 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.253726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.254017 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.254752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.256382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.263531 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.265275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.267415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1314/2612] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:11.271371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.281603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd005c51-b228-44bc-b1f2-6884c329d3b3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.286246 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbb56e-7b65-4afb-9bee-fd7978718d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.287122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbb56e-7b65-4afb-9bee-fd7978718d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.287412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbb56e-7b65-4afb-9bee-fd7978718d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.287692 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbb56e-7b65-4afb-9bee-fd7978718d44 admin admin] PATH_INFO: `/v3/projects/99171ff40c744344bb51facd3aac614e/users/2bd492d016e74cb4ba24639706b7ccbf/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.303206 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbb56e-7b65-4afb-9bee-fd7978718d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99171ff40c744344bb51facd3aac614e, user_id=2bd492d016e74cb4ba24639706b7ccbf, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.306098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbb56e-7b65-4afb-9bee-fd7978718d44 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.323794 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1315/2613] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:11 2022] PUT /identity/v3/projects/99171ff40c744344bb51facd3aac614e/users/2bd492d016e74cb4ba24639706b7ccbf/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:11.327564 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40983bed-92d2-418f-9bcb-98c5e52f370f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.334061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.334588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.334866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.368622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] MFA Rules not processed for user `2bd492d016e74cb4ba24639706b7ccbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:11.370013 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.371338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:11.405895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.406759 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.407194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.407654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.411188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.453863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1316/2614] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:11.455328 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.498787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.500235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.511237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1299/2615] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:11.549937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7713b7a-1f27-4b58-bd96-e7bfb22b1d13 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.555363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.556053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.556348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.556952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.557689 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.559626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.590723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1317/2616] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:11.594666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.606831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce7bf5-6716-4fa8-85cc-0d60cc8a4fe2 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.676783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.677552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.677872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.678158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.679444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.681610 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.706783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1300/2617] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:11.707956 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.720707 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545307c3-f2dc-40f9-b7af-14b9365a135e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.727978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.729100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.730853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.730853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.730853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.732201 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.738345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:11.752224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1318/2618] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:11.756322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.775338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff4da-f1c8-4dec-a95e-074faa78dcf1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.780536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.781455 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.781773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.782055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] PATH_INFO: `/v3/projects/75cd7db56d22466e97d88362026059c7/users/40301bdf368d486b814143ac4e0e563e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.799570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75cd7db56d22466e97d88362026059c7, user_id=40301bdf368d486b814143ac4e0e563e, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.802249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.817702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1301/2619] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:11 2022] PUT /identity/v3/projects/75cd7db56d22466e97d88362026059c7/users/40301bdf368d486b814143ac4e0e563e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:11.822800 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.833456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49c874-0cd4-4fe3-9aef-7ed6702a83e2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.910849 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d11f36-692f-4a89-83fd-dea05be1b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.911781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11f36-692f-4a89-83fd-dea05be1b782 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.912348 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11f36-692f-4a89-83fd-dea05be1b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.912636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11f36-692f-4a89-83fd-dea05be1b782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.914203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d11f36-692f-4a89-83fd-dea05be1b782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.916244 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d11f36-692f-4a89-83fd-dea05be1b782 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.933774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1319/2620] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:11.935238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:11.949385 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2938d-6176-46f1-a263-841415d71f80 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:11.955812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45950158-b0bb-4d5b-8633-765b583e825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:11.956849 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950158-b0bb-4d5b-8633-765b583e825d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.957262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950158-b0bb-4d5b-8633-765b583e825d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.957405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950158-b0bb-4d5b-8633-765b583e825d admin admin] PATH_INFO: `/v3/projects/75cd7db56d22466e97d88362026059c7/users/40301bdf368d486b814143ac4e0e563e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.959811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45950158-b0bb-4d5b-8633-765b583e825d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75cd7db56d22466e97d88362026059c7, user_id=40301bdf368d486b814143ac4e0e563e, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:11.963828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45950158-b0bb-4d5b-8633-765b583e825d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:11.969117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.969534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.969972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:11.983441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1302/2621] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:11 2022] PUT /identity/v3/projects/75cd7db56d22466e97d88362026059c7/users/40301bdf368d486b814143ac4e0e563e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:11.994596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:11.996020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:11.996602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:11.997883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:11.998350 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.034050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] MFA Rules not processed for user `40301bdf368d486b814143ac4e0e563e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:12.035300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.058669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1320/2622] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:12.066418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:12.112522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.114345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.125183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1303/2623] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:12.142211 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.145715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.202431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd3fc-fc78-4210-a974-f46e2d00b901 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.208018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.208685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.208975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.209258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.218785 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.220464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.227295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97b99-6709-46e7-8f19-ac0488756eda None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.229834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1321/2624] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:52:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:52:12.231464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062439d1-cb52-45e6-b7da-737aef4638b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.231994 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062439d1-cb52-45e6-b7da-737aef4638b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.232267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062439d1-cb52-45e6-b7da-737aef4638b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.233242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062439d1-cb52-45e6-b7da-737aef4638b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.244097 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062439d1-cb52-45e6-b7da-737aef4638b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.246081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062439d1-cb52-45e6-b7da-737aef4638b7 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.264768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1304/2625] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:12.344303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.395463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:12.436208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609ef78-475f-4117-b2f4-0b29a512c8f5 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.441147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.442167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.442545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.442871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] PATH_INFO: `/v3/users/2bd492d016e74cb4ba24639706b7ccbf` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.458332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd492d016e74cb4ba24639706b7ccbf)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:12.461401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:12.528522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd492d016e74cb4ba24639706b7ccbf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.529339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd492d016e74cb4ba24639706b7ccbf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.537868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd492d016e74cb4ba24639706b7ccbf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.538678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd492d016e74cb4ba24639706b7ccbf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.548585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd492d016e74cb4ba24639706b7ccbf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:12.577228 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1322/2626] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:12 2022] DELETE /identity/v3/users/2bd492d016e74cb4ba24639706b7ccbf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:12.578883 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.579175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.579557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:12.635356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:12.636648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.764915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1305/2627] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:12.775537 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:12.777835 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:12.954279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0ee77-6888-466b-a3fb-bd1834bf62af admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:12.968738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:12.969622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:12.969965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:12.970253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.004929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.012050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.068759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1323/2628] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:13.551737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.553483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:13.568622 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee7c62-c6f9-436b-852f-5cf38d705a92 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.574660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.575530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.575824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.576109 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] PATH_INFO: `/v3/projects/99171ff40c744344bb51facd3aac614e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.577324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=99171ff40c744344bb51facd3aac614e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.583000 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.660552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99171ff40c744344bb51facd3aac614e', 'request_id': 'req-0916fa1b-828a-467c-a707-2bffdaa999fa'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.668975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99171ff40c744344bb51facd3aac614e', 'request_id': 'req-0916fa1b-828a-467c-a707-2bffdaa999fa'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.671835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:13.678120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99171ff40c744344bb51facd3aac614e', 'request_id': 'req-0916fa1b-828a-467c-a707-2bffdaa999fa'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.701586 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] The token cache is being invalidate because project 99171ff40c744344bb51facd3aac614e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:13.702140 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.703050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.705250 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1306/2629] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:13 2022] DELETE /identity/v3/projects/99171ff40c744344bb51facd3aac614e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:13.709341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:13.730263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9f34-3ca2-4a26-b3c4-45cd75d834ca service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:13.796285 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:13.810770 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:13.811591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:13.811887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:13.812171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] PATH_INFO: `/v3/users/40301bdf368d486b814143ac4e0e563e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:13.813635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40301bdf368d486b814143ac4e0e563e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:13.815651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:13.880303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40301bdf368d486b814143ac4e0e563e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.881106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40301bdf368d486b814143ac4e0e563e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.890921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40301bdf368d486b814143ac4e0e563e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.892338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40301bdf368d486b814143ac4e0e563e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.902323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5895be9-feeb-4090-93d7-0205d3689f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40301bdf368d486b814143ac4e0e563e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:13.925517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1324/2630] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:13 2022] DELETE /identity/v3/users/40301bdf368d486b814143ac4e0e563e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:14.265584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.268231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:14.293621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916fa1b-828a-467c-a707-2bffdaa999fa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.427726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.429024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.429413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.429935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] PATH_INFO: `/v3/projects/75cd7db56d22466e97d88362026059c7` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.443521 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=75cd7db56d22466e97d88362026059c7)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.445815 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.521104 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75cd7db56d22466e97d88362026059c7', 'request_id': 'req-411c726b-d0d2-4a80-8e49-cae5847a8bda'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:14.531892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75cd7db56d22466e97d88362026059c7', 'request_id': 'req-411c726b-d0d2-4a80-8e49-cae5847a8bda'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:14.541430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75cd7db56d22466e97d88362026059c7', 'request_id': 'req-411c726b-d0d2-4a80-8e49-cae5847a8bda'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:14.572254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] The token cache is being invalidate because project 75cd7db56d22466e97d88362026059c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:14.572254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:14.572254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-411c726b-d0d2-4a80-8e49-cae5847a8bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:14.572254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1307/2631] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:14 2022] DELETE /identity/v3/projects/75cd7db56d22466e97d88362026059c7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:14.576396 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.576695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.577012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.597407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:14.599613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:14.682560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1325/2632] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:14.690975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.691364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.691720 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.723135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:14.724384 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:14.733040 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1308/2633] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:14.737449 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.751683 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3c06e-1a92-4a24-a0a9-20e4ee9b29ae None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.758871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.759796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.760258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.760552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.761481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:14.761855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.763994 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.779308 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1326/2634] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:14.783542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.794408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d74e08-a44b-4456-a3c7-198f36fcab0e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.799773 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.800530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.800889 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.801277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.802079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.803960 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.842392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1309/2635] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/projects => generated 401 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:14.843508 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.855862 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b29b-1af7-4eaf-a453-bfb5e3c2f394 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.924903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.925651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.925980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.926369 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.927190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.929941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:14.937638 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:14.958380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1327/2636] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:14 2022] POST /identity/v3/users => generated 301 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:14.962962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:14.974694 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb93ecf-4785-49bc-8240-a78ae30d13df admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:14.985432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:14.986170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:14.986467 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:14.986816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:14.987493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:14.989169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.012687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.018436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1310/2637] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:15.031026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ac218-8b8a-4a3c-bb21-acfcb4007cb0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.038621 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcc241-c9e9-4c52-a188-bc111c2825d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.039740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcc241-c9e9-4c52-a188-bc111c2825d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.040172 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcc241-c9e9-4c52-a188-bc111c2825d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.040639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcc241-c9e9-4c52-a188-bc111c2825d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.041814 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcc241-c9e9-4c52-a188-bc111c2825d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.044428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcc241-c9e9-4c52-a188-bc111c2825d4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.060667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1328/2638] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:15.064802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.078647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7b822-ecc4-4772-90f0-2b6b8d93ffbc admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.084584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.085801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.086266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.086551 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] PATH_INFO: `/v3/projects/834bc79cf4a445dc9405a784ba9ff51e/users/3901ea12f4c243eeaccc1d6a9d878cbe/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.107920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834bc79cf4a445dc9405a784ba9ff51e, user_id=3901ea12f4c243eeaccc1d6a9d878cbe, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.111353 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.128944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1311/2639] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:15 2022] PUT /identity/v3/projects/834bc79cf4a445dc9405a784ba9ff51e/users/3901ea12f4c243eeaccc1d6a9d878cbe/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:15.141587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.142399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.142657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.177064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] MFA Rules not processed for user `3901ea12f4c243eeaccc1d6a9d878cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:15.178472 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:15.267777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1329/2640] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:15.269539 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.298426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:15.370855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a514e4-5a6e-46fb-86b4-47db173cb045 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.377409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.378121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.378592 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.378881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.379738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.383267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.417914 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1312/2641] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:15 2022] POST /identity/v3/projects => generated 399 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:15.419238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.437146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1346a-3ebb-47de-81c8-ff3cfce439e4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.523486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.524364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.524709 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.525105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.526108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.528401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.556516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1330/2642] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:15 2022] POST /identity/v3/users => generated 306 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:15.558023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.570595 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16834c2c-de4f-459d-aaf3-333f26b1a3da admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.578225 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.578934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.579228 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.579515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.580282 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.582162 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.594795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1313/2643] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:52:15.599350 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.612977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5d807-903e-4d45-806f-6b96700b931e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.638189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.639211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.639593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.640310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] PATH_INFO: `/v3/projects/b7f2e5d7c4b3433cb6e4df5d1ab599cd/users/d2e83fb7b98e4fa2968be720d8b15064/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.677773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f2e5d7c4b3433cb6e4df5d1ab599cd, user_id=d2e83fb7b98e4fa2968be720d8b15064, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.682382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.712758 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1331/2644] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:15 2022] PUT /identity/v3/projects/b7f2e5d7c4b3433cb6e4df5d1ab599cd/users/d2e83fb7b98e4fa2968be720d8b15064/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:15.716544 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.729391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d214-bd66-49ee-b9c8-5b47d1ad8dd5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.821511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d743af-1aaf-4830-9d9b-84eab9eddc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.822687 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d743af-1aaf-4830-9d9b-84eab9eddc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.823024 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d743af-1aaf-4830-9d9b-84eab9eddc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.823589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d743af-1aaf-4830-9d9b-84eab9eddc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.824347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d743af-1aaf-4830-9d9b-84eab9eddc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.825999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d743af-1aaf-4830-9d9b-84eab9eddc92 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.841978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1314/2645] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:15.843463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:15.866131 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a2af4-45fa-4de1-90d9-d9eebbcc388e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:15.875804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:15.878026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.878325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.878611 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] PATH_INFO: `/v3/projects/b7f2e5d7c4b3433cb6e4df5d1ab599cd/users/d2e83fb7b98e4fa2968be720d8b15064/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.883585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f2e5d7c4b3433cb6e4df5d1ab599cd, user_id=d2e83fb7b98e4fa2968be720d8b15064, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:15.888432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:15.911634 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1332/2646] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:15 2022] PUT /identity/v3/projects/b7f2e5d7c4b3433cb6e4df5d1ab599cd/users/d2e83fb7b98e4fa2968be720d8b15064/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:15.914189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:15.914666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:15.915112 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:15.948325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] MFA Rules not processed for user `d2e83fb7b98e4fa2968be720d8b15064`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:15.949864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:16.107587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1315/2647] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:16.118844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:16.120659 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:16.172684 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:16.238465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336dba1-36d4-49ac-ba20-b03ac9f85102 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:16.246284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:16.246976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:16.247255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:16.247545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:16.260049 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:16.261520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:16.273443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1333/2648] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:16.383650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:16.395420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efee85c-7f2f-4019-94de-59255922b7e2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:16.399921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:16.400597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:16.400887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:16.401174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:16.411990 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:16.490898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:16.493213 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:16.508267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1316/2649] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:19.486935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:19.489635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:19.614897 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a09786-0fee-49f4-82bd-585564d50b86 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:19.620359 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:19.621132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:19.621528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:19.621942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:19.623907 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:19.768811 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:19.774158 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd7372-644f-4090-a38a-7529bc80848e service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:19.790023 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1334/2650] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:21.180333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:21.194015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0e506-7a64-4556-89f2-d69d5209dca1 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:21.199974 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:21.201138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:21.201607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:21.202095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] PATH_INFO: `/v3/users/3901ea12f4c243eeaccc1d6a9d878cbe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:21.203679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3901ea12f4c243eeaccc1d6a9d878cbe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:21.207788 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:21.286325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3901ea12f4c243eeaccc1d6a9d878cbe'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:21.315053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3901ea12f4c243eeaccc1d6a9d878cbe'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:21.330939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3901ea12f4c243eeaccc1d6a9d878cbe'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:21.346430 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3901ea12f4c243eeaccc1d6a9d878cbe'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:21.347370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3901ea12f4c243eeaccc1d6a9d878cbe'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:21.358859 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1317/2651] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:21 2022] DELETE /identity/v3/users/3901ea12f4c243eeaccc1d6a9d878cbe => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:21.360279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:21.360791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:21.360996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:21.381747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:21.383865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:21.532457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1335/2652] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:21.542847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:21.544320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:21.636190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09013-97af-4bd7-b6c4-84ade2623076 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:21.639995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:21.640669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:21.640961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:21.641249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:21.654503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:21.656105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:21.678393 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1318/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:22.035115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:22.040276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:22.115995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ff312-b599-4d40-bc37-5e61e09d9254 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:22.141646 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:22.142670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:22.143055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:22.143443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] PATH_INFO: `/v3/projects/834bc79cf4a445dc9405a784ba9ff51e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:22.145951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=834bc79cf4a445dc9405a784ba9ff51e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:22.148948 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:22.157924 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:22.233533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834bc79cf4a445dc9405a784ba9ff51e', 'request_id': 'req-2e659857-7e08-415b-988d-fd9242fcec8c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.244241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834bc79cf4a445dc9405a784ba9ff51e', 'request_id': 'req-2e659857-7e08-415b-988d-fd9242fcec8c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.252642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834bc79cf4a445dc9405a784ba9ff51e', 'request_id': 'req-2e659857-7e08-415b-988d-fd9242fcec8c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.266851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] The token cache is being invalidate because project 834bc79cf4a445dc9405a784ba9ff51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:22.267171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.267915 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.270871 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1336/2654] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:22 2022] DELETE /identity/v3/projects/834bc79cf4a445dc9405a784ba9ff51e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:22.278542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:22.300278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592cf9b-c7f5-4b34-8f21-cff7bc2d5aa8 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:22.335032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:22.408924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:22.410271 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:22.412623 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:22.413169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] PATH_INFO: `/v3/users/d2e83fb7b98e4fa2968be720d8b15064` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:22.416543 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e83fb7b98e4fa2968be720d8b15064)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:22.420242 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:22.528038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e83fb7b98e4fa2968be720d8b15064'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.557536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e83fb7b98e4fa2968be720d8b15064'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.572679 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e83fb7b98e4fa2968be720d8b15064'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.583888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e83fb7b98e4fa2968be720d8b15064'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.585364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8713bb55-706e-4430-9619-392d102c339b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e83fb7b98e4fa2968be720d8b15064'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:22.594769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1319/2655] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:22 2022] DELETE /identity/v3/users/d2e83fb7b98e4fa2968be720d8b15064 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:22.981485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:22.983382 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:23.019861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659857-7e08-415b-988d-fd9242fcec8c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:23.138606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:23.139980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.140286 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.145050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] PATH_INFO: `/v3/projects/b7f2e5d7c4b3433cb6e4df5d1ab599cd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.146426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f2e5d7c4b3433cb6e4df5d1ab599cd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:23.161845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:23.268805 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f2e5d7c4b3433cb6e4df5d1ab599cd', 'request_id': 'req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:23.282942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f2e5d7c4b3433cb6e4df5d1ab599cd', 'request_id': 'req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:23.293675 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f2e5d7c4b3433cb6e4df5d1ab599cd', 'request_id': 'req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:23.315579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] The token cache is being invalidate because project b7f2e5d7c4b3433cb6e4df5d1ab599cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:23.315928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:23.316822 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-df368c84-bb5b-4f3b-b3f8-e6b6e390d6a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:23.320309 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1337/2656] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:22 2022] DELETE /identity/v3/projects/b7f2e5d7c4b3433cb6e4df5d1ab599cd => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:23.325378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.325911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.328981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.357508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:23.359369 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:23.486961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1320/2657] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:23.496305 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.496656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.496874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.537149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:23.539243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:23.555690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1338/2658] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:23.555934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:23.572415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2054c0b-583b-4be9-ba8c-df33907c2f25 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:23.585313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:23.586051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.586344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.586636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.587402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:23.587722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:23.589811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:23.645173 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1321/2659] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:52:23.649874 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:23.661053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ff7d-8207-4d9d-a1b5-d7cc91c8029f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:23.681299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:23.682033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.682325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.683821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.684545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:23.687038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:23.727633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:23.755298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:23.756825 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1339/2660] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:23 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:23.776345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73ad2-66b5-4613-bbb5-1090a46450c4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:23.829803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:23.884205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:23.884978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.885324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.885632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.886526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:23.890124 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:23.925901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1322/2661] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:23 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:23.927341 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:23.962519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334773f5-a5b2-4b42-be29-e874e5331dc1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:23.971815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:23.974034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:23.974444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:23.975193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:23.975831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:23.978043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:23.996117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1340/2662] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:23.998458 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:24.012979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137c79-70ab-453d-b46f-3a48e50726ad admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:24.022795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f71e3-4daa-452d-aa94-ad2d22c591a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:24.023769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f71e3-4daa-452d-aa94-ad2d22c591a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.024135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f71e3-4daa-452d-aa94-ad2d22c591a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.024606 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f71e3-4daa-452d-aa94-ad2d22c591a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.025395 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f71e3-4daa-452d-aa94-ad2d22c591a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:24.027737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f71e3-4daa-452d-aa94-ad2d22c591a4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:24.046289 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1323/2663] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:24.055653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:24.067298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a9d6f-cba5-4a98-90e6-bc1af5acf83b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:24.075606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:24.076687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.077080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.077494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] PATH_INFO: `/v3/projects/51de59f6fa1d4ebeb50edaa162d19bd6/users/d1b141290c2248afb3fad29eb138f07c/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.098093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51de59f6fa1d4ebeb50edaa162d19bd6, user_id=d1b141290c2248afb3fad29eb138f07c, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:24.101084 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:24.121713 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1341/2664] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:24 2022] PUT /identity/v3/projects/51de59f6fa1d4ebeb50edaa162d19bd6/users/d1b141290c2248afb3fad29eb138f07c/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:24.132963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.132963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.132963 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.163397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] MFA Rules not processed for user `d1b141290c2248afb3fad29eb138f07c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:24.166406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.292721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1324/2665] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:24.304507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:24.306778 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:24.379398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca2111-3990-4bf4-b5ad-d35da4720be3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:24.386255 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:24.387089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:24.387485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:24.387872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:24.407284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:24.409147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:24.420512 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1342/2666] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:27.466050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:27.470842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:27.573847 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c6985-9297-4ca4-ac20-9fc46d2c568e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:27.577420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:27.578629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:27.578968 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:27.579257 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:27.581106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:27.612400 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:27.613976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:27.697293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:27.703875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:27.707070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:27.728448 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1325/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:27.755191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43524-3504-4e9c-bc33-d6c34befea1d service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:27.765387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:27.767343 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:27.767647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:27.768954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] PATH_INFO: `/v3/users/af5c10fc72dc44e59030216239fc678d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:27.771664 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5c10fc72dc44e59030216239fc678d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:27.775299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:27.849519 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5c10fc72dc44e59030216239fc678d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:27.850385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5c10fc72dc44e59030216239fc678d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:27.860548 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5c10fc72dc44e59030216239fc678d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:27.861549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5c10fc72dc44e59030216239fc678d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:27.874540 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5c10fc72dc44e59030216239fc678d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:27.880071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:27.898696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1343/2668] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:27 2022] DELETE /identity/v3/users/af5c10fc72dc44e59030216239fc678d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:28.134381 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:28.146413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998b2f2-f0bf-4aad-aa63-c51a0f1ce31b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:28.214776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:28.215733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:28.216345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:28.216345 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] PATH_INFO: `/v3/projects/79a3fa88f5ea4fa2940733b09cbc835a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:28.217487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a3fa88f5ea4fa2940733b09cbc835a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:28.219490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:28.282892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a3fa88f5ea4fa2940733b09cbc835a', 'request_id': 'req-2510783d-b213-4ee9-b48a-3fb415910b4d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.288462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a3fa88f5ea4fa2940733b09cbc835a', 'request_id': 'req-2510783d-b213-4ee9-b48a-3fb415910b4d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.294881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a3fa88f5ea4fa2940733b09cbc835a', 'request_id': 'req-2510783d-b213-4ee9-b48a-3fb415910b4d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.309101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] The token cache is being invalidate because project 79a3fa88f5ea4fa2940733b09cbc835a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:28.309403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.310062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.311699 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1326/2669] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:28 2022] DELETE /identity/v3/projects/79a3fa88f5ea4fa2940733b09cbc835a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:28.315660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:28.317054 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:28.331531 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16406f6e-81e6-442f-b3ed-d4ae9d53b44e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:28.391913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:28.392806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:28.393095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:28.393378 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] PATH_INFO: `/v3/users/9a8f02c2087d4048a8f6f3a0d91f3eca` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:28.394995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8f02c2087d4048a8f6f3a0d91f3eca)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:28.396806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:28.446454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8f02c2087d4048a8f6f3a0d91f3eca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.447463 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8f02c2087d4048a8f6f3a0d91f3eca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.461259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8f02c2087d4048a8f6f3a0d91f3eca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.462032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8f02c2087d4048a8f6f3a0d91f3eca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.469929 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-00f42e25-8f43-4427-81fd-1c00c1eada10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8f02c2087d4048a8f6f3a0d91f3eca'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.489179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1344/2670] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:28 2022] DELETE /identity/v3/users/9a8f02c2087d4048a8f6f3a0d91f3eca => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:28.684504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:28.686226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:28.706492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510783d-b213-4ee9-b48a-3fb415910b4d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:28.781036 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:28.782251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:28.782546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:28.782805 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] PATH_INFO: `/v3/projects/fab17e2c4af24819ad1de1b83ef2e464` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:28.783992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab17e2c4af24819ad1de1b83ef2e464)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:28.785950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:28.817344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:28.839794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab17e2c4af24819ad1de1b83ef2e464', 'request_id': 'req-b33fbf30-1596-40d3-b456-0190820f7d7e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.845715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab17e2c4af24819ad1de1b83ef2e464', 'request_id': 'req-b33fbf30-1596-40d3-b456-0190820f7d7e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.852246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab17e2c4af24819ad1de1b83ef2e464', 'request_id': 'req-b33fbf30-1596-40d3-b456-0190820f7d7e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.865023 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] The token cache is being invalidate because project fab17e2c4af24819ad1de1b83ef2e464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:28.865318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.865995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b33fbf30-1596-40d3-b456-0190820f7d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:28.867597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1327/2671] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:28 2022] DELETE /identity/v3/projects/fab17e2c4af24819ad1de1b83ef2e464 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:36.808588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:36.809602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:36.809729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:36.826885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] MFA Rules not processed for user `67d64cad336c44d28c9cf31fc9896854`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:36.828078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:36.881549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1345/2672] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:37.867660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:37.868183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:37.868292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:37.892056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] MFA Rules not processed for user `cd1fea13991f4f4b8369cfad704259b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:37.893401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:37.953728 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1328/2673] 10.222.0.57 () {58 vars in 967 bytes} [Wed Jun 15 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:37.967608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:37.969181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:37.981650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:38.047099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc79a1-7a63-46d7-a644-2e7797bfd6b3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:38.052133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf6e3c71de545d782c99ca87d86fb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:38.052790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:38.053081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:38.053363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:38.062314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:38.063845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:38.072745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1346/2674] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Jun 15 14:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:52:40.387207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:40.389094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:40.484560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e97ee-cb93-4bc4-8d7f-c317165343fd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:40.493445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:40.494498 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:40.494795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:40.495078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] PATH_INFO: `/v3/users/44f839b774b249dd9d7d3061feb2cd41` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:40.496349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f839b774b249dd9d7d3061feb2cd41)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:40.499183 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:40.558267 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f839b774b249dd9d7d3061feb2cd41'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.572632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:40.578318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f839b774b249dd9d7d3061feb2cd41'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.609952 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f839b774b249dd9d7d3061feb2cd41'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.618978 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f839b774b249dd9d7d3061feb2cd41'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.619922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-cf93d687-ee98-470b-a565-88e794a5bd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f839b774b249dd9d7d3061feb2cd41'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.630429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1329/2675] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:40 2022] DELETE /identity/v3/users/44f839b774b249dd9d7d3061feb2cd41 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:40.836816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:40.850303 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dffa3-8ebc-4500-a7cd-37f532b79fa9 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:40.915976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:40.917251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:40.917642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:40.918137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] PATH_INFO: `/v3/projects/8d40b771f30d4e668e0c908fbd0da517` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:40.919711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d40b771f30d4e668e0c908fbd0da517)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:40.922439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:40.983182 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d40b771f30d4e668e0c908fbd0da517', 'request_id': 'req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.992112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d40b771f30d4e668e0c908fbd0da517', 'request_id': 'req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:40.999424 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d40b771f30d4e668e0c908fbd0da517', 'request_id': 'req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.011591 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] The token cache is being invalidate because project 8d40b771f30d4e668e0c908fbd0da517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:41.011796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.013022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-95c36aec-bf5c-4b6f-a896-8f3fa5136ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:41.014772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1347/2676] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:40 2022] DELETE /identity/v3/projects/8d40b771f30d4e668e0c908fbd0da517 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:41.020950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.021251 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.021537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.040777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:41.042067 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:41.107470 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1330/2677] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:52:41.116034 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.116333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.116624 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.138578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:41.139823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:41.149723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1348/2678] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:41.154317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.164881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb18f4-da53-44a2-af46-4c82754321ed None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.170630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.171439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.171836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.172239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.173094 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:52:41.173549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.175322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.189148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1331/2679] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:52:41.193881 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.204040 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8c3a4-7e9f-464c-ae18-311801b08b1a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.208765 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.209433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.209760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.210051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.210797 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.212867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.240218 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:41.251936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1349/2680] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:52:41.256323 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.267913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b89b-2c50-4191-8afc-aa9d2d761961 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.328977 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.329686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.329979 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.330259 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.330967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.333545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.358445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1332/2681] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:52:41.359914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.370823 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65229eb-cbf1-4fae-b6b7-6f529b10f0c8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.383439 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.384432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.393081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.393440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.394300 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.395952 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.411704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1350/2682] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:41.413019 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.425290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244523-4387-42ce-a574-e5d67ba3221d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.430123 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e23d40-d5af-4f07-a8ee-afc4073b0ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.430813 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e23d40-d5af-4f07-a8ee-afc4073b0ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.431102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e23d40-d5af-4f07-a8ee-afc4073b0ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.431386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e23d40-d5af-4f07-a8ee-afc4073b0ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.432135 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e23d40-d5af-4f07-a8ee-afc4073b0ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.433782 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e23d40-d5af-4f07-a8ee-afc4073b0ec8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.445709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1333/2683] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:52:41.450412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.461516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026fe63-2ab4-4f17-b69b-23065051fd7f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.466500 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.467523 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.467812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.468095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] PATH_INFO: `/v3/projects/8628a713792a456eb6907a6cbb58210a/users/b1561c30f7134618a2563b618a1c4770/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.483486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8628a713792a456eb6907a6cbb58210a, user_id=b1561c30f7134618a2563b618a1c4770, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.486183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.500324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1351/2684] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:52:41 2022] PUT /identity/v3/projects/8628a713792a456eb6907a6cbb58210a/users/b1561c30f7134618a2563b618a1c4770/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:52:41.510651 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.510949 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.511239 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.540468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] MFA Rules not processed for user `b1561c30f7134618a2563b618a1c4770`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:41.541860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:41.579391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:41.612839 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1334/2685] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:41.624117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:41.625850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:41.684522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9a8a-61af-4701-b0f7-8e8a0564a78f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:41.688180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:41.688831 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:41.689117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:41.689397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:41.697625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:41.699073 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:41.708355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1352/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:51.609768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=12 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:51.704420 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:59.113802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:59.115240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.123080 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:59.124474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.186616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4dfac-d95d-48a1-9951-6340fe38200c service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:59.187127 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b46a-2edf-4a09-9bb4-0ebf322b400c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:59.190481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:59.191143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.191433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.191719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:59.192126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:59.193014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.193329 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.193565 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.194320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] PATH_INFO: `/v3/users/d1b141290c2248afb3fad29eb138f07c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:59.195333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b141290c2248afb3fad29eb138f07c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:59.197253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:59.255726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b141290c2248afb3fad29eb138f07c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.270901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b141290c2248afb3fad29eb138f07c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.275827 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:59.277438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1f8e8-824d-42e7-8e4e-8ab0245fd6b4 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:59.279597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b141290c2248afb3fad29eb138f07c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.288744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b141290c2248afb3fad29eb138f07c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.289584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b141290c2248afb3fad29eb138f07c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.289755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1353/2687] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:52:59.296383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1335/2688] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:52:59 2022] DELETE /identity/v3/users/d1b141290c2248afb3fad29eb138f07c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:59.301134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.301433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.301752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:59.320062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:59.321289 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.387754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1354/2689] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:59.398689 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:59.400766 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.464134 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba605db-d2c7-4603-9be3-bd47108adcf3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:59.467896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b79a4-da13-46df-91c8-742101eba3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:59.468540 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b79a4-da13-46df-91c8-742101eba3ef service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.468823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b79a4-da13-46df-91c8-742101eba3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.469103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b79a4-da13-46df-91c8-742101eba3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:59.479185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b79a4-da13-46df-91c8-742101eba3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:59.480863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b79a4-da13-46df-91c8-742101eba3ef service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:59.490325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1336/2690] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:52:59.775340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:52:59.777356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.792021 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801467d9-390d-4fe9-95dd-cec1ccea28ca None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:52:59.798168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:52:59.799266 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.799682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.800063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] PATH_INFO: `/v3/projects/51de59f6fa1d4ebeb50edaa162d19bd6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:59.801498 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51de59f6fa1d4ebeb50edaa162d19bd6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:52:59.803861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:52:59.857419 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:59.860154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51de59f6fa1d4ebeb50edaa162d19bd6', 'request_id': 'req-ab1a7b2c-e109-4155-82dd-89299587e936'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.868795 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51de59f6fa1d4ebeb50edaa162d19bd6', 'request_id': 'req-ab1a7b2c-e109-4155-82dd-89299587e936'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.874969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51de59f6fa1d4ebeb50edaa162d19bd6', 'request_id': 'req-ab1a7b2c-e109-4155-82dd-89299587e936'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.887595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] The token cache is being invalidate because project 51de59f6fa1d4ebeb50edaa162d19bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:52:59.887895 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.888616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab1a7b2c-e109-4155-82dd-89299587e936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:52:59.890243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1355/2691] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:52:59 2022] DELETE /identity/v3/projects/51de59f6fa1d4ebeb50edaa162d19bd6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:52:59.895245 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.895541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.895826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:52:59.916090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:52:59.917334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:52:59.934600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:52:59.982273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1337/2692] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:52:59.990185 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:52:59.990812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:52:59.990812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.010024 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:00.011271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.019528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1356/2693] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:00.024351 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.034230 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3edfe0-03cf-4271-b4f7-0095efe672ef None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.045117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.045866 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.046151 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.046428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.047161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:00.047472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.049075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.061744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1338/2694] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:53:00.066239 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.075954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bd93f-a312-432c-8bc0-0829350a8842 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.081087 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.081787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.082076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.082363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.083189 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.085191 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.115338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1357/2695] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:00 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:00.117403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.127420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4519747-dd2e-4d92-a21f-815daa9205ce admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.183970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.184648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.184936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.185217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.185985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.187900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.216556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1339/2696] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:00 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:00.218335 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.232100 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a73485-0356-40fe-b06c-cb635a34c9df admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.237858 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.238555 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.238846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.239132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.239896 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.241697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.255760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1358/2697] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:00.257753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.269031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c97d72-e38a-4620-90a8-fa41353515cf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.274769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.275491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.275786 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.276075 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.276818 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.278530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.289178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1340/2698] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:00.293853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.305582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e8d13-80ae-480f-834a-458813f9274b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.310438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce577-92e1-4367-a91c-249ea8cd6758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.311495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce577-92e1-4367-a91c-249ea8cd6758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.311966 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce577-92e1-4367-a91c-249ea8cd6758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.312195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce577-92e1-4367-a91c-249ea8cd6758 admin admin] PATH_INFO: `/v3/projects/87c354ee56e9479f8b65a0f83c4c8050/users/d9c5bf292ee74c718d7b1f332858b44e/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.323506 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.325325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.330691 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ce577-92e1-4367-a91c-249ea8cd6758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c354ee56e9479f8b65a0f83c4c8050, user_id=d9c5bf292ee74c718d7b1f332858b44e, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.334089 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ce577-92e1-4367-a91c-249ea8cd6758 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.339768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6146cdc-05b6-4df5-810d-22d49737b446 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.345174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.346372 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.346967 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.347461 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] PATH_INFO: `/v3/users/b1561c30f7134618a2563b618a1c4770` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.348861 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1561c30f7134618a2563b618a1c4770)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.351158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.353851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1359/2699] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:00 2022] PUT /identity/v3/projects/87c354ee56e9479f8b65a0f83c4c8050/users/d9c5bf292ee74c718d7b1f332858b44e/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:00.364578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.366302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.366302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.395798 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] MFA Rules not processed for user `d9c5bf292ee74c718d7b1f332858b44e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:00.397943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.413779 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1561c30f7134618a2563b618a1c4770'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:00.426584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1561c30f7134618a2563b618a1c4770'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:00.433991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1561c30f7134618a2563b618a1c4770'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:00.443188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1561c30f7134618a2563b618a1c4770'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:00.444660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fac5b1a0-9be1-4a33-b9f7-cb946d14aefe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1561c30f7134618a2563b618a1c4770'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:00.449494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:00.460268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1341/2700] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:00 2022] DELETE /identity/v3/users/b1561c30f7134618a2563b618a1c4770 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:00.464742 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.465246 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.465628 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.477196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1360/2701] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:00.481864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.483250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.489590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:00.491439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.562323 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7c181-cefb-4019-b0a5-f44ff43d3a9c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.568326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:00.571406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1342/2702] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:00.583229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.585888 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:00.639599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.640582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.640878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.641166 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.642041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.643993 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.652110 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed71c3-93e1-4521-affd-46d9bc426c16 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.656858 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.657158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.657442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.657761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.672835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.674326 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.676606 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1361/2703] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:00 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:00.681230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.687076 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1343/2704] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:00.755636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c80fd-037b-47a2-a8c4-90aca5221de8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.761123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.761753 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.762365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.763775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.763775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.766123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.788996 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1362/2705] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:00 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:00.792844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.805561 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c259d67-8e34-41f5-9188-b483751dc76f service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.810832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.811658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.812400 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.812535 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.813277 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.814933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.832233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1344/2706] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:00.834385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.849124 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e12fb-a38d-41b3-9aa4-a7df208c24b9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.854541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.855701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.855992 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.856271 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] PATH_INFO: `/v3/projects/c4411b6fb2d04ff68b868c056c6c1770/users/308791777d384654958efb9c0c814c5c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.876014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4411b6fb2d04ff68b868c056c6c1770, user_id=308791777d384654958efb9c0c814c5c, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.878799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.897837 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1363/2707] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:00 2022] PUT /identity/v3/projects/c4411b6fb2d04ff68b868c056c6c1770/users/308791777d384654958efb9c0c814c5c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:00.901961 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:00.914505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5159ad-8da0-4fea-928c-71a56231d35e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:00.983139 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:00.983940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:00.984333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:00.984713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:00.985713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:00.987536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:00.999175 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1345/2708] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:01.003398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:01.020442 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:01.021091 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718f200-00f5-41f3-aee2-1c8591e02b34 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:01.024370 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:01.027429 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51581774-23e9-4494-9bd7-222eaa91e499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:01.028355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51581774-23e9-4494-9bd7-222eaa91e499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:01.028648 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51581774-23e9-4494-9bd7-222eaa91e499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:01.028933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51581774-23e9-4494-9bd7-222eaa91e499 admin admin] PATH_INFO: `/v3/projects/c4411b6fb2d04ff68b868c056c6c1770/users/308791777d384654958efb9c0c814c5c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:01.036872 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51581774-23e9-4494-9bd7-222eaa91e499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4411b6fb2d04ff68b868c056c6c1770, user_id=308791777d384654958efb9c0c814c5c, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:01.039951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51581774-23e9-4494-9bd7-222eaa91e499 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:01.043149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7fb14-0fb2-4dbd-8022-184a7870bb4a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:01.050922 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:01.051820 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:01.052105 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:01.053029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] PATH_INFO: `/v3/projects/8628a713792a456eb6907a6cbb58210a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:01.054792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8628a713792a456eb6907a6cbb58210a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:01.056898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:01.062850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1364/2709] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:01 2022] PUT /identity/v3/projects/c4411b6fb2d04ff68b868c056c6c1770/users/308791777d384654958efb9c0c814c5c/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:01.064406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:01.064775 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:01.065127 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:01.088959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:01.100355 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] MFA Rules not processed for user `308791777d384654958efb9c0c814c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:01.101734 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:01.138647 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8628a713792a456eb6907a6cbb58210a', 'request_id': 'req-56ee618f-7969-49ec-8280-2d878344d781'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:01.150452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:01.151226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8628a713792a456eb6907a6cbb58210a', 'request_id': 'req-56ee618f-7969-49ec-8280-2d878344d781'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:01.162817 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8628a713792a456eb6907a6cbb58210a', 'request_id': 'req-56ee618f-7969-49ec-8280-2d878344d781'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:01.181934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] The token cache is being invalidate because project 8628a713792a456eb6907a6cbb58210a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:01.182507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:01.183197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:01.185303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1346/2710] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:01 2022] DELETE /identity/v3/projects/8628a713792a456eb6907a6cbb58210a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:01.231137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1365/2711] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:01.242332 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:01.243796 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:01.400141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee618f-7969-49ec-8280-2d878344d781 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:01.412052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:01.414542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:01.415178 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:01.416526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:01.420609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:01.527096 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:01.528956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9718c-0d07-45be-8210-bc1cb342b272 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:01.539681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1347/2712] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:04.170898 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:04.172470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:04.236806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef69199-ae9a-4702-8e99-f7081ad2fea1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:04.241376 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:04.242310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:04.242593 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:04.242877 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] PATH_INFO: `/v3/users/d9c5bf292ee74c718d7b1f332858b44e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:04.244064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c5bf292ee74c718d7b1f332858b44e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:04.246032 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:04.293062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c5bf292ee74c718d7b1f332858b44e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.293739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c5bf292ee74c718d7b1f332858b44e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.300630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c5bf292ee74c718d7b1f332858b44e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.301290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c5bf292ee74c718d7b1f332858b44e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.309324 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c5bf292ee74c718d7b1f332858b44e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.326726 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1366/2713] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:04 2022] DELETE /identity/v3/users/d9c5bf292ee74c718d7b1f332858b44e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:04.331187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:04.331675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:04.331783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:04.350399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:04.352368 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:04.411483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1348/2714] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:04.422095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:04.423597 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:04.479001 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:04.483326 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755b817-74f7-4fef-8f76-3bc527be8cff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:04.487911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:04.488630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:04.489147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:04.489639 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:04.499121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:04.500545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:04.510433 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1367/2715] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:04.779216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:04.780983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:04.793640 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a27bd8-73aa-4b52-bc48-8c46ae11bc68 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:04.798305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:04.799180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:04.799474 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:04.799757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] PATH_INFO: `/v3/projects/87c354ee56e9479f8b65a0f83c4c8050` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:04.801029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c354ee56e9479f8b65a0f83c4c8050)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:04.802983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:04.804898 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:04.854130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c354ee56e9479f8b65a0f83c4c8050', 'request_id': 'req-76b3fb62-cb1a-4d80-933b-75b7c8228362'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.859876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c354ee56e9479f8b65a0f83c4c8050', 'request_id': 'req-76b3fb62-cb1a-4d80-933b-75b7c8228362'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.866435 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c354ee56e9479f8b65a0f83c4c8050', 'request_id': 'req-76b3fb62-cb1a-4d80-933b-75b7c8228362'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.879298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] The token cache is being invalidate because project 87c354ee56e9479f8b65a0f83c4c8050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:04.879596 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.880231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:04.881827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1349/2716] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:04 2022] DELETE /identity/v3/projects/87c354ee56e9479f8b65a0f83c4c8050 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:04.885941 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:04.895522 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f00b14-61d2-40b9-ab88-6cd016c1218d service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:04.952028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:04.952826 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:04.953114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:04.953397 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] PATH_INFO: `/v3/users/308791777d384654958efb9c0c814c5c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:04.954776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308791777d384654958efb9c0c814c5c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:04.956697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.002687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308791777d384654958efb9c0c814c5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.003339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308791777d384654958efb9c0c814c5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.010514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308791777d384654958efb9c0c814c5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.011167 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308791777d384654958efb9c0c814c5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.018039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52839a01-909e-4166-a109-d0e34b71e0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308791777d384654958efb9c0c814c5c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.036325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1368/2717] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:04 2022] DELETE /identity/v3/users/308791777d384654958efb9c0c814c5c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:05.306574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.308013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:05.321337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3fb62-cb1a-4d80-933b-75b7c8228362 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.378753 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.379633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.379927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.380210 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] PATH_INFO: `/v3/projects/c4411b6fb2d04ff68b868c056c6c1770` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.381546 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4411b6fb2d04ff68b868c056c6c1770)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.384070 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.432190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4411b6fb2d04ff68b868c056c6c1770', 'request_id': 'req-d7a1c875-16f8-4de1-9c69-87982bdcbafc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.437626 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4411b6fb2d04ff68b868c056c6c1770', 'request_id': 'req-d7a1c875-16f8-4de1-9c69-87982bdcbafc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.443828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4411b6fb2d04ff68b868c056c6c1770', 'request_id': 'req-d7a1c875-16f8-4de1-9c69-87982bdcbafc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.459276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] The token cache is being invalidate because project c4411b6fb2d04ff68b868c056c6c1770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:05.459574 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.460249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d7a1c875-16f8-4de1-9c69-87982bdcbafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:05.462528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1350/2718] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:05 2022] DELETE /identity/v3/projects/c4411b6fb2d04ff68b868c056c6c1770 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:05.467582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.468026 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.468165 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.486319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:05.487486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:05.548647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1369/2719] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:05.557445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.557777 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.558089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.576653 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:05.577903 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:05.585851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1351/2720] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:05.590053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.599588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b68f80-57e0-4791-96c1-24781be8d760 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.604293 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.604955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.605250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.605532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.606571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:05.606883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.608481 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.615232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:05.622299 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1370/2721] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:05.626736 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.636733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f88d6-85f7-47ee-ac93-45d733beb484 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.641399 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.642090 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.642379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.642658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.643357 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.645243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.655198 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:05.671875 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1352/2722] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:05 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:05.676058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.686864 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90369158-3a73-4257-8cac-4c7b46fffce4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.743144 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.743880 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.744204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.744492 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.745180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.747115 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.766944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1371/2723] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:05 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:05.771209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.781443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d73ee-1386-4b10-a27e-57ed8764f080 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.786202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.786892 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.787185 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.787468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.788205 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.789865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.800077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1353/2724] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:05.803887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.814863 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee01f2-f933-4520-abd4-061671be1336 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.819468 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde47b73-7f6b-49bb-81d3-5be071d5733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.820117 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde47b73-7f6b-49bb-81d3-5be071d5733f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.820991 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde47b73-7f6b-49bb-81d3-5be071d5733f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.821283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde47b73-7f6b-49bb-81d3-5be071d5733f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.822237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde47b73-7f6b-49bb-81d3-5be071d5733f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.823824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde47b73-7f6b-49bb-81d3-5be071d5733f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.833628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1372/2725] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:05.837864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:05.849615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818e1a2-72df-473e-a026-387c5e9f8d32 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:05.854418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:05.855298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.855585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.856120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] PATH_INFO: `/v3/projects/cfca50d2ec844fb2abbb7f38dd41ec7b/users/3f4d173188d94544981a13937e00cbae/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.871318 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfca50d2ec844fb2abbb7f38dd41ec7b, user_id=3f4d173188d94544981a13937e00cbae, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:05.873985 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:05.887739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1354/2726] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:05 2022] PUT /identity/v3/projects/cfca50d2ec844fb2abbb7f38dd41ec7b/users/3f4d173188d94544981a13937e00cbae/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:05.898076 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:05.898374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:05.898657 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:05.926020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] MFA Rules not processed for user `3f4d173188d94544981a13937e00cbae`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:05.927281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:05.991451 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1373/2727] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:05.996997 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.059932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b05543-5b3f-4a08-8c87-fb7be0b2c8c0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.064672 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.065337 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.065630 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.065953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.066659 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.068556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.092682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1355/2728] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:05 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:06.096936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.109095 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d701c-b1a9-4765-9599-9a7b35cc3301 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.164790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.165517 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.165843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.166133 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.166830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.168784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.178307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:06.190533 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1374/2729] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:06 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:06.194483 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.204980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eaa2da-cef1-4067-874e-43a46379dd6f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.209789 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.210502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.210802 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.211083 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.211877 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.213797 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.224491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1356/2730] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:06.228958 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.239097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b66d4e-185b-4509-bc0e-32d9b2032343 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.244327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c38990-96fb-4726-8b8f-75e29de7e5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.244985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c38990-96fb-4726-8b8f-75e29de7e5de admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.245275 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c38990-96fb-4726-8b8f-75e29de7e5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.245560 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c38990-96fb-4726-8b8f-75e29de7e5de admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.246354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c38990-96fb-4726-8b8f-75e29de7e5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.247995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c38990-96fb-4726-8b8f-75e29de7e5de admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.257754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1375/2731] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:06.261987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.271800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd7ce2-ca5e-4d63-bb19-f28df215c68f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.277334 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.278240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.278530 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.278809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] PATH_INFO: `/v3/projects/8f138223fd524af29dedf3118f28d42b/users/9aa89bd1b9a54817bfc865456acc4384/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.293878 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f138223fd524af29dedf3118f28d42b, user_id=9aa89bd1b9a54817bfc865456acc4384, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.296494 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.309838 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1357/2732] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:06 2022] PUT /identity/v3/projects/8f138223fd524af29dedf3118f28d42b/users/9aa89bd1b9a54817bfc865456acc4384/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:06.376050 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01799dac-980b-4624-98bf-33587764b630 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.376340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01799dac-980b-4624-98bf-33587764b630 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.376622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01799dac-980b-4624-98bf-33587764b630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.404592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-01799dac-980b-4624-98bf-33587764b630 None None] MFA Rules not processed for user `9aa89bd1b9a54817bfc865456acc4384`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:06.405828 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-01799dac-980b-4624-98bf-33587764b630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:06.470916 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1376/2733] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:06.475712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.536857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fde8f-a459-4f5d-86ea-695506048f0f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.542491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.543149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.543436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.543719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.544421 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.546366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.555913 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:06.571951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1358/2734] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:06 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:06.575953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01799dac-980b-4624-98bf-33587764b630 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.587317 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01799dac-980b-4624-98bf-33587764b630 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.642971 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.643672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.643963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.644245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.644945 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.646912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.667064 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1377/2735] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:06 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:53:06.671018 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.681169 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab2670-4d64-4c94-b268-ea70e0ad7648 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.685823 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.686485 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.686776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.687057 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.687787 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.689377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.700292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1359/2736] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:06.704226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.715356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ea91a-5a48-4bfb-b83a-f582090801a5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.720105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.721109 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.721401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.721705 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] PATH_INFO: `/v3/projects/cd599221885c4d21a92d343214812d77/users/9836d28ee465412b8db64b5c721294c8/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.737290 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd599221885c4d21a92d343214812d77, user_id=9836d28ee465412b8db64b5c721294c8, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.739964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.752789 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1378/2737] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:06 2022] PUT /identity/v3/projects/cd599221885c4d21a92d343214812d77/users/9836d28ee465412b8db64b5c721294c8/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:06.756744 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.767522 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c421579-5620-444d-967b-03de1f597856 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.824996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb5bec-3ce2-41d9-be3b-226206d408a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.825589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5bec-3ce2-41d9-be3b-226206d408a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.826327 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5bec-3ce2-41d9-be3b-226206d408a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.826615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5bec-3ce2-41d9-be3b-226206d408a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.827346 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb5bec-3ce2-41d9-be3b-226206d408a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.828956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb5bec-3ce2-41d9-be3b-226206d408a8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.839660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1360/2738] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:53:06.844426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:06.854724 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63360bd2-2bdc-4808-8d94-82e052c3d823 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:06.859578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:06.860490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.860780 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.861067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] PATH_INFO: `/v3/projects/cd599221885c4d21a92d343214812d77/users/9836d28ee465412b8db64b5c721294c8/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.863082 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd599221885c4d21a92d343214812d77, user_id=9836d28ee465412b8db64b5c721294c8, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:06.865730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:06.875846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1379/2739] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:06 2022] PUT /identity/v3/projects/cd599221885c4d21a92d343214812d77/users/9836d28ee465412b8db64b5c721294c8/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:06.887118 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:06.887408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:06.887757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:06.921950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] MFA Rules not processed for user `9836d28ee465412b8db64b5c721294c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:06.923160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:06.990146 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1361/2740] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:07.001530 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:07.003229 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:07.034178 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:07.061954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd7822-f37f-4645-871c-b72fe65edb9a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:07.065520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:07.066190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:07.066477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:07.066754 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:07.122511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:07.123869 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:07.132552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1380/2741] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:08.609010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:08.610413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:08.622790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ab6c5-17ab-490b-9202-bbbd427320ff None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:08.626348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:08.627001 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:08.627290 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:08.627575 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:08.683085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:08.684444 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:08.689085 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:08.694752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1362/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:09.237174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:09.246132 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d4cba-02e9-49be-a00c-e0b7496cd8a3 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:09.249784 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1086d-d5f8-40af-a3ee-2b57023c68d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:09.250436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1086d-d5f8-40af-a3ee-2b57023c68d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:09.250731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1086d-d5f8-40af-a3ee-2b57023c68d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:09.251015 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1086d-d5f8-40af-a3ee-2b57023c68d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:09.260494 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1086d-d5f8-40af-a3ee-2b57023c68d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:09.261950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1086d-d5f8-40af-a3ee-2b57023c68d6 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:09.271307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1381/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:13.094180 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:13.155970 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922e001-397e-44dd-ad64-5946f7de790c service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:13.160495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:13.161366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.161654 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.161964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] PATH_INFO: `/v3/users/3f4d173188d94544981a13937e00cbae` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.163157 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4d173188d94544981a13937e00cbae)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:13.165065 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:13.210464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4d173188d94544981a13937e00cbae'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.223053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4d173188d94544981a13937e00cbae'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.230490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4d173188d94544981a13937e00cbae'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.237707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4d173188d94544981a13937e00cbae'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.238322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4d173188d94544981a13937e00cbae'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.244197 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1363/2744] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:13 2022] DELETE /identity/v3/users/3f4d173188d94544981a13937e00cbae => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:13.248738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.249219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.249329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.267042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:13.268252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:13.327114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1382/2745] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:13.336829 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:13.339060 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:13.396004 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc27e5-10d2-4683-9b5b-28a6d5d7b719 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:13.400660 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:13.401331 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.401621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.401932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.410722 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:13.412131 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:13.422025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1364/2746] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:13.689128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:13.690577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:13.703534 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef36b6-e582-4616-81b8-3c026f318016 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:13.708446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:13.709431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.709839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.710193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] PATH_INFO: `/v3/projects/cfca50d2ec844fb2abbb7f38dd41ec7b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.713152 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfca50d2ec844fb2abbb7f38dd41ec7b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:13.715108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:13.766307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfca50d2ec844fb2abbb7f38dd41ec7b', 'request_id': 'req-ea65814b-ba85-424e-885a-c5e50f8a2a44'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.773448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfca50d2ec844fb2abbb7f38dd41ec7b', 'request_id': 'req-ea65814b-ba85-424e-885a-c5e50f8a2a44'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.778258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:13.779488 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfca50d2ec844fb2abbb7f38dd41ec7b', 'request_id': 'req-ea65814b-ba85-424e-885a-c5e50f8a2a44'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.792599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] The token cache is being invalidate because project cfca50d2ec844fb2abbb7f38dd41ec7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:13.792917 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.793556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.795122 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1383/2747] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:13 2022] DELETE /identity/v3/projects/cfca50d2ec844fb2abbb7f38dd41ec7b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:53:13.799174 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:13.809905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c1203-492e-44c9-af2a-a3b0d7659d30 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:13.828717 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:13.867809 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:13.869095 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:13.869380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:13.869740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] PATH_INFO: `/v3/users/9aa89bd1b9a54817bfc865456acc4384` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:13.870887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa89bd1b9a54817bfc865456acc4384)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:13.872801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:13.919010 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa89bd1b9a54817bfc865456acc4384'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.931940 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa89bd1b9a54817bfc865456acc4384'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.939649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa89bd1b9a54817bfc865456acc4384'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.947760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa89bd1b9a54817bfc865456acc4384'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.948386 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa89bd1b9a54817bfc865456acc4384'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:13.954281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1365/2748] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:13 2022] DELETE /identity/v3/users/9aa89bd1b9a54817bfc865456acc4384 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:14.233876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:14.235423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:14.250771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65814b-ba85-424e-885a-c5e50f8a2a44 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:14.313761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:14.314631 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:14.314919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:14.315200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] PATH_INFO: `/v3/projects/8f138223fd524af29dedf3118f28d42b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:14.316387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f138223fd524af29dedf3118f28d42b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:14.318590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:14.364666 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f138223fd524af29dedf3118f28d42b', 'request_id': 'req-ab460aab-7214-4ffa-ac91-206d6bfc05af'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.372978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f138223fd524af29dedf3118f28d42b', 'request_id': 'req-ab460aab-7214-4ffa-ac91-206d6bfc05af'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.378656 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f138223fd524af29dedf3118f28d42b', 'request_id': 'req-ab460aab-7214-4ffa-ac91-206d6bfc05af'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.389461 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] The token cache is being invalidate because project 8f138223fd524af29dedf3118f28d42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:14.389781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.390399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.391930 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1384/2749] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:14 2022] DELETE /identity/v3/projects/8f138223fd524af29dedf3118f28d42b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:14.395894 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:14.397322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:14.410240 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d1cd8-44ee-41b8-81f1-588b2c8d4eb4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:14.467303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:14.468190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:14.468473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:14.468754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] PATH_INFO: `/v3/users/9836d28ee465412b8db64b5c721294c8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:14.469959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9836d28ee465412b8db64b5c721294c8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:14.471851 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:14.515020 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:14.518079 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9836d28ee465412b8db64b5c721294c8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.532398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9836d28ee465412b8db64b5c721294c8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.539149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9836d28ee465412b8db64b5c721294c8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.546991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9836d28ee465412b8db64b5c721294c8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.547619 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c55230b3-c556-4f82-ad3d-1059ce412903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9836d28ee465412b8db64b5c721294c8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.553527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1366/2750] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:14 2022] DELETE /identity/v3/users/9836d28ee465412b8db64b5c721294c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:14.826687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:14.828389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:14.841813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab460aab-7214-4ffa-ac91-206d6bfc05af admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:14.899180 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:14.900136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:14.900445 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:14.900729 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] PATH_INFO: `/v3/projects/cd599221885c4d21a92d343214812d77` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:14.901942 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd599221885c4d21a92d343214812d77)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:14.903870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:14.927995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:14.951712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd599221885c4d21a92d343214812d77', 'request_id': 'req-de790b68-0b9c-4897-91bb-ccc3bbb46c36'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.961961 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd599221885c4d21a92d343214812d77', 'request_id': 'req-de790b68-0b9c-4897-91bb-ccc3bbb46c36'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.968965 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd599221885c4d21a92d343214812d77', 'request_id': 'req-de790b68-0b9c-4897-91bb-ccc3bbb46c36'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.982325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] The token cache is being invalidate because project cd599221885c4d21a92d343214812d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:14.982617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.983256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-de790b68-0b9c-4897-91bb-ccc3bbb46c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:14.984793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1385/2751] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:14 2022] DELETE /identity/v3/projects/cd599221885c4d21a92d343214812d77 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:14.989783 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:14.990087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:14.990379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.008981 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:15.010322 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:15.070403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1367/2752] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:15.079310 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.079616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.079900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.099297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:15.100507 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:15.109203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1386/2753] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:15.113512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.123803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec27c59-3d48-4a5a-95cd-5ade4be3f629 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.128365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.129021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.129307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.129589 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.130344 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:53:15.130656 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.132238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.143716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1368/2754] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:53:15.147936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.156957 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381934f2-f894-43d3-bdb5-b33cb77453b8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.161531 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.162219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.162509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.162791 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.163490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.165917 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.190990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1387/2755] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:15.195314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.205633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434207c0-56b9-45bf-97da-84bef3678979 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.261800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.262475 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.262768 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.263055 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.263752 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.265675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.285287 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1369/2756] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:53:15.289356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.300106 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b33956-8256-4d7c-91bb-a05f5f058219 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.304669 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.305346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.305637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.305968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.306702 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.308302 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.319074 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1388/2757] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:15.323515 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.333590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0676b-e50e-41f0-ba66-60f5c76a3f2c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.339668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab28c12-36bc-4da7-b115-b7b3fe4b0981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.340342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab28c12-36bc-4da7-b115-b7b3fe4b0981 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.340634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab28c12-36bc-4da7-b115-b7b3fe4b0981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.340920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab28c12-36bc-4da7-b115-b7b3fe4b0981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.341675 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab28c12-36bc-4da7-b115-b7b3fe4b0981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.343298 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab28c12-36bc-4da7-b115-b7b3fe4b0981 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.354054 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1370/2758] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:53:15.358636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.368842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f99673-024b-4936-bf11-2c82368682d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.373640 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.374554 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.374845 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.375128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] PATH_INFO: `/v3/projects/c9153b9b8af246488b18a0fa4c396b93/users/d9ff325786f3438f992fcb10fb2453f8/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.389925 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9153b9b8af246488b18a0fa4c396b93, user_id=d9ff325786f3438f992fcb10fb2453f8, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.392558 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.405853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1389/2759] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:53:15 2022] PUT /identity/v3/projects/c9153b9b8af246488b18a0fa4c396b93/users/d9ff325786f3438f992fcb10fb2453f8/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:15.416821 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.417293 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.417415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.443453 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] MFA Rules not processed for user `d9ff325786f3438f992fcb10fb2453f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:15.444696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:15.458217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:15.512983 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1371/2760] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:15.524574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:15.526876 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:15.582520 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81900651-11ba-4b48-b65e-4b0b2c6a8737 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:15.586147 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ca2cd515aa84a3ca01656977036ac07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:15.586793 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:15.587676 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:15.587928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:15.596313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:15.597766 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:15.607536 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1390/2761] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:16.190707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:16.192181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:16.250273 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0a144-9e69-487b-8552-91428a5098d7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:16.253801 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:16.254446 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:16.254731 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:16.255009 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:16.256754 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:16.334578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:16.337324 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7df79d-7a75-49a1-becf-6e884e14aa39 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:16.350086 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1372/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:17.754954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:17.756489 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:17.814012 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acacb8f-1965-4bff-be8a-3a1ae470fa40 service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:17.817592 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:17.818284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:17.818574 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:17.818851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:17.820643 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:17.830815 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:17.835761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:17.838330 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:17.851282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1391/2763] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:26.345417 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=74 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:27.846069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:31.474776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:31.475272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:31.475396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:31.495299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] MFA Rules not processed for user `6caa64aecb914e968d057b6b593f667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:31.496527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:31.552795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1373/2764] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:53:34.046844 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:34.048331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:34.059155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ef4c7-c3e3-4a3a-93a8-0bdf42bd1c38 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:34.063963 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:34.064619 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:34.064903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:34.065181 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:34.067011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:34.078731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:34.080247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83798dc4-077a-4e39-b7b3-bbc43a0a74ef service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:34.089325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1392/2765] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:41.545363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=25 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:44.085099 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:50.934956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:50.999084 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584157da-d2f3-41d9-8323-7b0a347a81c8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.003956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.004850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.005142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.005428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] PATH_INFO: `/v3/users/d9ff325786f3438f992fcb10fb2453f8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.006665 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ff325786f3438f992fcb10fb2453f8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.008579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.058027 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ff325786f3438f992fcb10fb2453f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.070998 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ff325786f3438f992fcb10fb2453f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.078524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ff325786f3438f992fcb10fb2453f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.086339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ff325786f3438f992fcb10fb2453f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.086958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ff325786f3438f992fcb10fb2453f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.093305 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1374/2766] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:53:50 2022] DELETE /identity/v3/users/d9ff325786f3438f992fcb10fb2453f8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:53:51.098363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.098851 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.098953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.116098 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:51.117314 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.177176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1393/2767] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:51.187007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:51.189026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.252559 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fc012-0074-4ff0-8ce1-949e5b0817e2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.256161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.256980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.257283 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.257590 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.273371 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.274842 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.285418 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1375/2768] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:51.566740 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:51.568693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:51.585438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344fd7f-3d39-4325-801a-85e04ea15d7b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:51.591019 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:51.591900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:51.592188 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:51.592467 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] PATH_INFO: `/v3/projects/c9153b9b8af246488b18a0fa4c396b93` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:51.593718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9153b9b8af246488b18a0fa4c396b93)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:51.596327 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:51.646006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9153b9b8af246488b18a0fa4c396b93', 'request_id': 'req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.653976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9153b9b8af246488b18a0fa4c396b93', 'request_id': 'req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.661352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9153b9b8af246488b18a0fa4c396b93', 'request_id': 'req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.674190 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] The token cache is being invalidate because project c9153b9b8af246488b18a0fa4c396b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:53:51.674412 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.675063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f36fa6a9-229d-4cee-a2fb-907bdd9198b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:53:51.676626 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1394/2769] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:53:51 2022] DELETE /identity/v3/projects/c9153b9b8af246488b18a0fa4c396b93 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:54:01.280304 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:01.670057 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:41.770416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:41.771886 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:41.865927 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7e2d8-acd6-4fd7-a2e5-08c2a6ef45e5 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:41.956931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:41.958026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:41.958549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:41.959038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:41.961101 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:41.979191 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:41.980832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:41.995272 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1376/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:45.325645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:45.326575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:45.326575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:45.345075 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] MFA Rules not processed for user `6caa64aecb914e968d057b6b593f667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:54:45.346603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:45.403454 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1395/2771] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:54:46.170150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:46.171609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.231120 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623dc59-61e8-4fec-b1f5-c8db6512e132 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:46.235281 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:46.235931 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:46.236216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:46.236495 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:46.238297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.248698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:46.250155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:46.259769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1377/2772] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:46.816102 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:46.818283 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.893526 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:46.895268 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.913134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af9d3b-3cd4-4b87-a4c0-9a3e35e853cb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:46.918233 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f170f42-8f7c-4394-88b8-f0e0155dd753 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:46.919245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:46.919919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:46.920211 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:46.920495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:46.922440 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.925951 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:46.926733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:46.927117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:46.927691 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:46.930413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:46.936301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:46.938086 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:46.948868 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:46.950762 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:46.951371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1396/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:46.970089 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1378/2774] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:56.944183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=57 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:56.964243 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=94 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:57.830065 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:54:57.845598 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f86ee2-0cab-4bf3-9d66-1177f6ac593d service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:54:57.851474 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:54:57.852149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:54:57.852444 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:54:57.852727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:54:57.854843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:54:57.870745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:54:57.872473 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:54:57.883919 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1397/2775] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:54:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:02.938247 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:02.948320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657432d1-f0b6-4303-908e-b8024beb8c80 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:02.952677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:02.953336 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:02.953617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:02.953944 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:02.955995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:03.024936 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:03.026403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:03.036723 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1379/2776] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:06.118108 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:06.126846 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:06.141356 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55425a4b-322f-4586-95ba-2366bbffe731 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:06.149578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:06.150332 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:06.150690 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:06.151051 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:06.153128 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:06.166867 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:06.168913 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:06.179806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1398/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:09.042781 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:09.044207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:09.056098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d237d-65d3-43f5-b60b-59c230825ce2 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:09.061535 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:09.062462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:09.062751 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:09.063053 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] PATH_INFO: `/v3/users/67d64cad336c44d28c9cf31fc9896854` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:09.064234 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d64cad336c44d28c9cf31fc9896854)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:09.066189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:09.112831 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d64cad336c44d28c9cf31fc9896854'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.125081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d64cad336c44d28c9cf31fc9896854'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.132285 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d64cad336c44d28c9cf31fc9896854'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.140708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d64cad336c44d28c9cf31fc9896854'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.141352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d64cad336c44d28c9cf31fc9896854'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.147279 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1380/2778] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:09 2022] DELETE /identity/v3/users/67d64cad336c44d28c9cf31fc9896854 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:09.323387 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:09.348267 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a789-693b-489d-876f-c1686463b441 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:09.405651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:09.406547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:09.406861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:09.407142 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] PATH_INFO: `/v3/projects/2350f7b62a7d4d8199faf8b18b2ca8dc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:09.408333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2350f7b62a7d4d8199faf8b18b2ca8dc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:09.410284 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:09.463977 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2350f7b62a7d4d8199faf8b18b2ca8dc', 'request_id': 'req-6f51206c-09ee-4779-a195-95c396586929'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.472910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2350f7b62a7d4d8199faf8b18b2ca8dc', 'request_id': 'req-6f51206c-09ee-4779-a195-95c396586929'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.479493 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2350f7b62a7d4d8199faf8b18b2ca8dc', 'request_id': 'req-6f51206c-09ee-4779-a195-95c396586929'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.491423 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] The token cache is being invalidate because project 2350f7b62a7d4d8199faf8b18b2ca8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:55:09.491719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.492383 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.493951 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1399/2779] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:09 2022] DELETE /identity/v3/projects/2350f7b62a7d4d8199faf8b18b2ca8dc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:09.498347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:09.499792 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:09.513908 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676fd6e-8e51-43d9-aae3-0fe1275b2c9f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:09.573160 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:09.574130 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:09.574420 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:09.574702 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] PATH_INFO: `/v3/users/d7775b4643f049808494ea2b0efecc6d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:09.575939 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7775b4643f049808494ea2b0efecc6d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:09.577934 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:09.583671 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:09.628356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7775b4643f049808494ea2b0efecc6d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.641472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7775b4643f049808494ea2b0efecc6d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.648537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7775b4643f049808494ea2b0efecc6d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.657364 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7775b4643f049808494ea2b0efecc6d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.658021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7775b4643f049808494ea2b0efecc6d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:09.664708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1381/2780] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:09 2022] DELETE /identity/v3/users/d7775b4643f049808494ea2b0efecc6d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:09.871988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:09.873432 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:09.888759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51206c-09ee-4779-a195-95c396586929 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:09.951727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:09.952617 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:09.952910 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:09.953195 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] PATH_INFO: `/v3/projects/3e1f7cd040e246329d53abe443bb6b9a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:09.954402 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1f7cd040e246329d53abe443bb6b9a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:09.956297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:09.962469 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:10.016660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1f7cd040e246329d53abe443bb6b9a', 'request_id': 'req-eab2f145-c852-43cb-ba54-a8ff7ed9c056'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:10.024137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1f7cd040e246329d53abe443bb6b9a', 'request_id': 'req-eab2f145-c852-43cb-ba54-a8ff7ed9c056'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:10.030184 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1f7cd040e246329d53abe443bb6b9a', 'request_id': 'req-eab2f145-c852-43cb-ba54-a8ff7ed9c056'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:10.041282 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] The token cache is being invalidate because project 3e1f7cd040e246329d53abe443bb6b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:55:10.041509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:10.042168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:10.044243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1400/2781] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:09 2022] DELETE /identity/v3/projects/3e1f7cd040e246329d53abe443bb6b9a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:19.661041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:20.038241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:37.669456 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:37.671043 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:37.792158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d552-b97f-4ca2-bc5d-90e5a8bfa8cd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:37.854087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5953aa40eb54d399b593000f3c74596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:37.854737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:37.855025 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:37.855307 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:37.857414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:37.901311 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:37.914834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:37.916397 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:37.925607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1382/2782] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:47.921608 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:57.537813 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:57.539262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:57.637757 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f145-c852-43cb-ba54-a8ff7ed9c056 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:57.641502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:57.642187 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:57.642478 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:57.642759 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:57.645242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:57.727693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:57.729045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:57.747752 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1401/2783] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:00.649253 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:00.650921 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:00.668072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fb94f-872d-4796-9090-5600181151a7 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:00.675181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:00.676226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.676545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.676826 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] PATH_INFO: `/v3/users/6caa64aecb914e968d057b6b593f667f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.678092 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6caa64aecb914e968d057b6b593f667f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:00.680068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.726844 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6caa64aecb914e968d057b6b593f667f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.739676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6caa64aecb914e968d057b6b593f667f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.746528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6caa64aecb914e968d057b6b593f667f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.754464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6caa64aecb914e968d057b6b593f667f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.755102 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6fc0ac01-f57d-4063-91d7-7de8f64c45e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6caa64aecb914e968d057b6b593f667f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.761032 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1383/2784] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:00 2022] DELETE /identity/v3/users/6caa64aecb914e968d057b6b593f667f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:00.931192 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:00.941110 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6687194-d331-4c0e-9db8-180990b435bf service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:00.992937 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:00.999503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.000377 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.000667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.001068 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] PATH_INFO: `/v3/projects/591d111357224477998b265d45e3ee0d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.002297 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591d111357224477998b265d45e3ee0d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.004386 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.050585 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591d111357224477998b265d45e3ee0d', 'request_id': 'req-b897f95e-3696-44ed-a07f-0bb7b1839242'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:01.057727 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591d111357224477998b265d45e3ee0d', 'request_id': 'req-b897f95e-3696-44ed-a07f-0bb7b1839242'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:01.063173 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591d111357224477998b265d45e3ee0d', 'request_id': 'req-b897f95e-3696-44ed-a07f-0bb7b1839242'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:01.073782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] The token cache is being invalidate because project 591d111357224477998b265d45e3ee0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:01.074071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:01.074695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b897f95e-3696-44ed-a07f-0bb7b1839242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:01.076248 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1402/2785] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:00 2022] DELETE /identity/v3/projects/591d111357224477998b265d45e3ee0d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:01.081737 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.082117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.082410 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.138126 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.139295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.198617 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1384/2786] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.206897 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.207193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.207485 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.226203 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.227415 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.235524 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1403/2787] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.240098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.249899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbf1c3-4f30-4d35-a4f0-3d2cbc83c187 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.255676 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.256338 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.256648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.256926 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.257677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:01.258002 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.259604 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.271856 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1385/2788] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:01.276159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.285528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea47a-5b5c-451a-970a-0ac24a5eb5cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.290222 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.290878 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.291171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.291457 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.292183 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.294114 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.319840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1404/2789] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:01.323930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.334578 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3ddeb-d873-4eff-bbf7-0d192257e99c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.362313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:01.398871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.399502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.399816 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.400128 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.400873 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.402836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.422706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1386/2790] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:01.427287 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.437777 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bed9e-f4ce-48b9-9385-d998047f04cf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.442905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.443569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.443861 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.444146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.444887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.446535 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.458233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1405/2791] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:01.462376 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.473045 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8ed7-a85d-4f4f-9812-9d491fdd504f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.479871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef31c-47b6-4a51-979f-ded52dfdbe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.480677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef31c-47b6-4a51-979f-ded52dfdbe05 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.480964 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef31c-47b6-4a51-979f-ded52dfdbe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.481373 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef31c-47b6-4a51-979f-ded52dfdbe05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.482394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef31c-47b6-4a51-979f-ded52dfdbe05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.484008 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef31c-47b6-4a51-979f-ded52dfdbe05 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.495865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1387/2792] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:01.500839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.511944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4c0a8-f753-400a-afb5-70c0e7870711 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.516551 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.517436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.517751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.518859 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] PATH_INFO: `/v3/projects/83d235970dad43e0acd46bbb87add850/users/5bcbe09091c54ee98168e4e31e0403a4/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.534563 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d235970dad43e0acd46bbb87add850, user_id=5bcbe09091c54ee98168e4e31e0403a4, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.537247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.550776 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1406/2793] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:01 2022] PUT /identity/v3/projects/83d235970dad43e0acd46bbb87add850/users/5bcbe09091c54ee98168e4e31e0403a4/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:01.560709 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.561103 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.561479 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.589843 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] MFA Rules not processed for user `5bcbe09091c54ee98168e4e31e0403a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.591066 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.658447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1388/2794] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.662901 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.728224 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f53d14-8b73-4e99-be9e-52b8562f6cc3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.733088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.733796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.734090 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.734502 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.735208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.737215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.763120 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1407/2795] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:01.767082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.778748 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317fea-3fb7-4c1d-9c66-8c1767423b34 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.836649 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.837320 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.837607 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.837932 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.838629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.840524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.860147 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1389/2796] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:01.865088 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.875399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271336f-9e38-4e9b-b30c-0c9536e70ccf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.880653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.881318 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.881613 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.881959 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.882697 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.884291 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.886385 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:01.900541 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1408/2797] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:01.901946 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.912039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c02ef0-78c0-4169-99e0-1055b3044c16 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.916615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.917503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.917824 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.918113 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] PATH_INFO: `/v3/projects/2ba67b18e38c4a19a448c5c5386af305/users/0e8bcb774ef540959ef5d9eaaa543c2e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.934433 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba67b18e38c4a19a448c5c5386af305, user_id=0e8bcb774ef540959ef5d9eaaa543c2e, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.937100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.950950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1390/2798] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:01 2022] PUT /identity/v3/projects/2ba67b18e38c4a19a448c5c5386af305/users/0e8bcb774ef540959ef5d9eaaa543c2e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:01.954947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.965042 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1893d-a153-47ac-9474-2243581ee085 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.022083 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6d4bc-987a-4f75-921f-bb5edeecb341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.022756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d4bc-987a-4f75-921f-bb5edeecb341 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.023046 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d4bc-987a-4f75-921f-bb5edeecb341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.023329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d4bc-987a-4f75-921f-bb5edeecb341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.024059 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6d4bc-987a-4f75-921f-bb5edeecb341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.025685 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6d4bc-987a-4f75-921f-bb5edeecb341 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.035980 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1409/2799] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:02.040171 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.045580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:02.052342 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d48682-06ca-4dd7-9cc2-7251616b99cf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.057477 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.058403 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.058830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.059116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] PATH_INFO: `/v3/projects/2ba67b18e38c4a19a448c5c5386af305/users/0e8bcb774ef540959ef5d9eaaa543c2e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.061150 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba67b18e38c4a19a448c5c5386af305, user_id=0e8bcb774ef540959ef5d9eaaa543c2e, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.063794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.074541 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1391/2800] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:02 2022] PUT /identity/v3/projects/2ba67b18e38c4a19a448c5c5386af305/users/0e8bcb774ef540959ef5d9eaaa543c2e/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:02.086258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.086557 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.086865 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.115404 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] MFA Rules not processed for user `0e8bcb774ef540959ef5d9eaaa543c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:02.116599 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:02.184340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1410/2801] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:02.196514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.198236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:02.277632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfb384-743c-45f4-a77f-8f8587ed24f0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.281392 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.282082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.282377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.282662 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.292296 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.293761 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.304317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1392/2802] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:02.375245 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.385020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ae57-18ce-4c55-b9ef-b667656578c1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.388608 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11138216-43e5-404b-bdc0-d9a720bb9651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.389254 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11138216-43e5-404b-bdc0-d9a720bb9651 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.389547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11138216-43e5-404b-bdc0-d9a720bb9651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.389870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11138216-43e5-404b-bdc0-d9a720bb9651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.447588 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11138216-43e5-404b-bdc0-d9a720bb9651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.448939 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11138216-43e5-404b-bdc0-d9a720bb9651 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.459509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1411/2803] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:12.298986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:12.454279 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:40.579528 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:40.643719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d076a-7585-4469-8396-3bca01f5dcc8 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:40.648865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:40.649864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:40.650156 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:40.650437 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] PATH_INFO: `/v3/users/5bcbe09091c54ee98168e4e31e0403a4` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:40.651639 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcbe09091c54ee98168e4e31e0403a4)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:40.653579 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:40.698629 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcbe09091c54ee98168e4e31e0403a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.710812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcbe09091c54ee98168e4e31e0403a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.717347 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcbe09091c54ee98168e4e31e0403a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.725339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcbe09091c54ee98168e4e31e0403a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.725999 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcbe09091c54ee98168e4e31e0403a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.732138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1393/2804] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:40 2022] DELETE /identity/v3/users/5bcbe09091c54ee98168e4e31e0403a4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:40.736647 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:40.737136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:40.737240 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:40.754200 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:40.755428 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:40.812773 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1412/2805] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:40.822992 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:40.824387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:40.880445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f67265-9015-48d4-a0ce-1377fe5600a8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:40.884064 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:40.884708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:40.884995 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:40.885295 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:40.894276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:40.896396 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:40.905706 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1394/2806] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:41.168860 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:41.170325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:41.182595 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5026490-12ea-4ce5-99f1-4c57be5b97af None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:41.187583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:41.188564 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.188900 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.189252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] PATH_INFO: `/v3/projects/83d235970dad43e0acd46bbb87add850` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.190582 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d235970dad43e0acd46bbb87add850)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:41.194006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:41.241995 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d235970dad43e0acd46bbb87add850', 'request_id': 'req-d13559be-4731-4a29-aa4e-e293d214f20a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.249067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d235970dad43e0acd46bbb87add850', 'request_id': 'req-d13559be-4731-4a29-aa4e-e293d214f20a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.254587 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d235970dad43e0acd46bbb87add850', 'request_id': 'req-d13559be-4731-4a29-aa4e-e293d214f20a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.266041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] The token cache is being invalidate because project 83d235970dad43e0acd46bbb87add850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:41.266334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.266975 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.268506 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1413/2807] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:41 2022] DELETE /identity/v3/projects/83d235970dad43e0acd46bbb87add850 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:41.273199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:41.282677 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981860b-fb2d-4780-8385-441eb8e9865e service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:41.300408 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:41.342314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:41.343642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.344077 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.344508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] PATH_INFO: `/v3/users/0e8bcb774ef540959ef5d9eaaa543c2e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.346313 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8bcb774ef540959ef5d9eaaa543c2e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:41.349446 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:41.399549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8bcb774ef540959ef5d9eaaa543c2e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.411030 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8bcb774ef540959ef5d9eaaa543c2e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.418405 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8bcb774ef540959ef5d9eaaa543c2e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.425711 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8bcb774ef540959ef5d9eaaa543c2e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.426321 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-111a7a81-d588-4fd7-952a-5a88047ed9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8bcb774ef540959ef5d9eaaa543c2e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.433297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1395/2808] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:41 2022] DELETE /identity/v3/users/0e8bcb774ef540959ef5d9eaaa543c2e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:41.709038 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:41.710569 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:41.728964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13559be-4731-4a29-aa4e-e293d214f20a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:41.801833 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:41.802988 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.803273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.803552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] PATH_INFO: `/v3/projects/2ba67b18e38c4a19a448c5c5386af305` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.804730 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba67b18e38c4a19a448c5c5386af305)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:41.806688 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:41.837462 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:41.855922 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba67b18e38c4a19a448c5c5386af305', 'request_id': 'req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.863516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba67b18e38c4a19a448c5c5386af305', 'request_id': 'req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.869829 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba67b18e38c4a19a448c5c5386af305', 'request_id': 'req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.881486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] The token cache is being invalidate because project 2ba67b18e38c4a19a448c5c5386af305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:41.881806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.882471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-556b0eae-5bcb-48a0-85c6-5dba7cd2d52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:41.884006 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1414/2809] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:41 2022] DELETE /identity/v3/projects/2ba67b18e38c4a19a448c5c5386af305 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:41.889959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.890441 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.890537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.908472 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:41.909697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:41.969975 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1396/2810] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:41.978204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.978503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.978821 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.998020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:41.999252 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.007742 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1415/2811] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:42.011988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.022464 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc4b34-c7ca-4dd3-a698-6d453262b677 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.027058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.027727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.028072 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.028363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.029125 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:42.029445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.031087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.042716 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1397/2812] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:42.046955 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.056812 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8c8a5-b344-43d5-bc8d-374f1a326cc7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.061645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.062334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.062650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.062932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.063738 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.065682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.091121 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1416/2813] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:42.096416 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.106415 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986957-cda2-4e89-9760-8c1b7312c22c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.133190 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:42.166443 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.167129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.167419 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.167707 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.168413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.170365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.190516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1398/2814] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:42.194756 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.204935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e69a-769a-4662-99d5-1d595514ac4d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.210148 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.210806 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.211130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.211417 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.212150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.213769 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.224586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1417/2815] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:42.228542 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.238666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416528-b60e-4ef0-983a-667ec461882b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.244142 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a9adf-8c93-4e9c-a347-42ebbc0046aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.244812 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a9adf-8c93-4e9c-a347-42ebbc0046aa admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.245099 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a9adf-8c93-4e9c-a347-42ebbc0046aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.245380 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a9adf-8c93-4e9c-a347-42ebbc0046aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.246158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a9adf-8c93-4e9c-a347-42ebbc0046aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.247746 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a9adf-8c93-4e9c-a347-42ebbc0046aa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.257869 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1399/2816] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:42.261855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.272635 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cfbc6-5a8e-42b9-949f-baa6c81a716e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.277258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.278171 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.278465 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.278748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] PATH_INFO: `/v3/projects/f7149e22f7e4468c90c9a1a87d0490d7/users/dc0716116e1b4a538b4943ee972c380f/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.293809 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7149e22f7e4468c90c9a1a87d0490d7, user_id=dc0716116e1b4a538b4943ee972c380f, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.296436 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.309143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1418/2817] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:42 2022] PUT /identity/v3/projects/f7149e22f7e4468c90c9a1a87d0490d7/users/dc0716116e1b4a538b4943ee972c380f/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:42.319189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.319489 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.319776 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.346791 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] MFA Rules not processed for user `dc0716116e1b4a538b4943ee972c380f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:42.348088 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.412231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1400/2818] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:42.416667 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.482176 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b61c7-667d-432e-8f35-0724868068b1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.486751 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.487406 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.487693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.487969 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.488653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.491854 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.516329 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1419/2819] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:42.520445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.531274 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8385a-ad95-4fa9-b761-4a2eb2d070f2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.589778 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.590388 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.590681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.590966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.591668 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.593583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.614511 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1401/2820] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:42.618779 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.629135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69f71-dc47-4e51-9425-bdbb078494b4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.634693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.635354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.635658 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.635921 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.636649 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.638263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.649272 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1420/2821] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:42.653315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.663552 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957107a4-4738-425f-83b8-910f6b800afb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.668138 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.669017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.669301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.669581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] PATH_INFO: `/v3/projects/45a121ec07b044a6a16396cc34f05c46/users/76f0c5d9987c4deb80cb575c8da9ab82/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.684708 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a121ec07b044a6a16396cc34f05c46, user_id=76f0c5d9987c4deb80cb575c8da9ab82, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.687379 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.700462 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1402/2822] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:42 2022] PUT /identity/v3/projects/45a121ec07b044a6a16396cc34f05c46/users/76f0c5d9987c4deb80cb575c8da9ab82/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:42.704723 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.715990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c24b9c-8da9-4e42-8196-387fe754a260 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.736018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:42.774179 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192dc68-da6f-4c3f-871b-49a0c7383fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.774847 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192dc68-da6f-4c3f-871b-49a0c7383fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.775134 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192dc68-da6f-4c3f-871b-49a0c7383fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.775414 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192dc68-da6f-4c3f-871b-49a0c7383fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.776136 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192dc68-da6f-4c3f-871b-49a0c7383fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.777741 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192dc68-da6f-4c3f-871b-49a0c7383fc6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.788237 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1421/2823] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:42.792432 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.797915 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:42.805129 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e756ff-f19c-49a8-aa51-cffa1d8695e4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.809988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.810865 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.811149 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.811426 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] PATH_INFO: `/v3/projects/45a121ec07b044a6a16396cc34f05c46/users/76f0c5d9987c4deb80cb575c8da9ab82/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.813359 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a121ec07b044a6a16396cc34f05c46, user_id=76f0c5d9987c4deb80cb575c8da9ab82, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.816214 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.826635 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1403/2824] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:42 2022] PUT /identity/v3/projects/45a121ec07b044a6a16396cc34f05c46/users/76f0c5d9987c4deb80cb575c8da9ab82/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:42.838030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.838483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.838594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.866884 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] MFA Rules not processed for user `76f0c5d9987c4deb80cb575c8da9ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:42.868028 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.935375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1422/2825] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:42.946429 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.948199 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.007897 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833960e-d7bb-4465-be53-1df351c37372 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.011570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.012232 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.012524 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.012810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.022402 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.023828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.033367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1404/2826] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:43.080842 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.082340 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.094824 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc01e8-80f4-49a6-b8b2-377869f97ece None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.098459 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4a52-b339-43b1-93ab-dbef6d494dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.099687 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4a52-b339-43b1-93ab-dbef6d494dbb service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.099976 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4a52-b339-43b1-93ab-dbef6d494dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.100256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4a52-b339-43b1-93ab-dbef6d494dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.154987 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4a52-b339-43b1-93ab-dbef6d494dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.156345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4a52-b339-43b1-93ab-dbef6d494dbb service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.165747 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1423/2827] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:46.392473 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:46.455835 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e6c6-a042-4bb6-82eb-a21168472a1b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:46.460537 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:46.461407 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:46.461726 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:46.462014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] PATH_INFO: `/v3/users/dc0716116e1b4a538b4943ee972c380f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:46.463215 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0716116e1b4a538b4943ee972c380f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:46.465133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:46.511584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0716116e1b4a538b4943ee972c380f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:46.523114 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0716116e1b4a538b4943ee972c380f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:46.529958 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0716116e1b4a538b4943ee972c380f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:46.538582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0716116e1b4a538b4943ee972c380f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:46.539436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0716116e1b4a538b4943ee972c380f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:46.545335 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1405/2828] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:46 2022] DELETE /identity/v3/users/dc0716116e1b4a538b4943ee972c380f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:46.549986 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:46.550464 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:46.550589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:46.568325 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:46.569514 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:46.608974 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:46.627421 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1424/2829] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:46.639648 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:46.641068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:46.655976 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:46.710680 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19e20-f362-41c0-bb9b-44a4502a4c20 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:46.715569 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:46.716300 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:46.716585 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:46.716863 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:46.726962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:46.728378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:46.737870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1406/2830] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:47.037883 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:47.040105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:47.062063 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a72072-f252-47ab-9c45-b0c9a2f984ec None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:47.068882 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:47.069808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.070097 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.070375 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] PATH_INFO: `/v3/projects/f7149e22f7e4468c90c9a1a87d0490d7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.071623 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7149e22f7e4468c90c9a1a87d0490d7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:47.073525 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:47.124259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7149e22f7e4468c90c9a1a87d0490d7', 'request_id': 'req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.133322 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7149e22f7e4468c90c9a1a87d0490d7', 'request_id': 'req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.140363 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7149e22f7e4468c90c9a1a87d0490d7', 'request_id': 'req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.155247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] The token cache is being invalidate because project f7149e22f7e4468c90c9a1a87d0490d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:47.155545 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.156209 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.157783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1425/2831] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:47 2022] DELETE /identity/v3/projects/f7149e22f7e4468c90c9a1a87d0490d7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:47.161980 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:47.172525 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5c1d-4907-4e35-8710-c6e0ef4eaa70 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:47.248155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:47.249238 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.249560 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.249882 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] PATH_INFO: `/v3/users/76f0c5d9987c4deb80cb575c8da9ab82` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.251143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f0c5d9987c4deb80cb575c8da9ab82)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:47.253082 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:47.305512 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f0c5d9987c4deb80cb575c8da9ab82'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.319492 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f0c5d9987c4deb80cb575c8da9ab82'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.326352 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f0c5d9987c4deb80cb575c8da9ab82'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.337056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f0c5d9987c4deb80cb575c8da9ab82'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.337855 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f32ef292-87fe-46ea-b8ab-ca44eb5e1644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f0c5d9987c4deb80cb575c8da9ab82'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.346911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1407/2832] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:47 2022] DELETE /identity/v3/users/76f0c5d9987c4deb80cb575c8da9ab82 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:47.658622 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:47.660204 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:47.675455 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f7934-ecf4-4e10-9cd1-8f17208464fa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:47.734196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:47.735067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.735354 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.735636 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] PATH_INFO: `/v3/projects/45a121ec07b044a6a16396cc34f05c46` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.736839 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a121ec07b044a6a16396cc34f05c46)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:47.738790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:47.787247 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a121ec07b044a6a16396cc34f05c46', 'request_id': 'req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.794589 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a121ec07b044a6a16396cc34f05c46', 'request_id': 'req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.800150 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a121ec07b044a6a16396cc34f05c46', 'request_id': 'req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.808273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:47.814207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] The token cache is being invalidate because project 45a121ec07b044a6a16396cc34f05c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:47.814567 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.815490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a5bf472-c3f2-42e5-b4bf-ab7591ae1eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:47.817928 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1426/2833] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:47 2022] DELETE /identity/v3/projects/45a121ec07b044a6a16396cc34f05c46 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:47.823007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.823504 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.823613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.841571 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:47.842822 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:47.873390 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:47.908026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1408/2834] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:47.917632 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.917954 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.918236 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.937364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:47.938600 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:47.946703 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1427/2835] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:47.950871 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:47.960891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a6e0-fdd1-426c-883c-a465674fbf0e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:47.965775 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:47.966447 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.966738 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.967021 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.967760 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:47.968078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:47.969683 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:47.981891 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1409/2836] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:47.986196 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:47.996672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a009f-c99d-40ab-a297-ca64b50e2ad0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.001431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.002130 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.005112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.005112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.005112 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.005450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.030119 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1428/2837] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:47 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:48.034261 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.044953 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07054327-a348-4ae3-b7fb-f36822171e90 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.100732 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.101406 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.101727 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.102017 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.102715 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.104621 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.124161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1410/2838] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:48 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:48.128253 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.138816 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518042b-149d-461e-b839-eea1edf913d5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.143586 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.144251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.144542 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.144834 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.145556 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.147781 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.158241 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1429/2839] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:48.162905 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.172769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1913db9-ae0b-4a64-88b3-d27fcbb41a20 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.177613 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e4e4e-7c65-44b9-8f8c-c7af98fb237f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.178306 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e4e4e-7c65-44b9-8f8c-c7af98fb237f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.178593 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e4e4e-7c65-44b9-8f8c-c7af98fb237f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.178872 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e4e4e-7c65-44b9-8f8c-c7af98fb237f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.179600 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e4e4e-7c65-44b9-8f8c-c7af98fb237f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.181202 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e4e4e-7c65-44b9-8f8c-c7af98fb237f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.191181 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1411/2840] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:48.195510 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.205256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f8b77-7de6-4b87-a5e8-eff8af190d36 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.210487 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.211362 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.211652 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.211933 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] PATH_INFO: `/v3/projects/db5e8bb8d8c8484f8b9a0c147f287b4e/users/06d4d54297384d11acd3142e1f5e714b/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.227603 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5e8bb8d8c8484f8b9a0c147f287b4e, user_id=06d4d54297384d11acd3142e1f5e714b, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.230273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.243256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1430/2841] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:48 2022] PUT /identity/v3/projects/db5e8bb8d8c8484f8b9a0c147f287b4e/users/06d4d54297384d11acd3142e1f5e714b/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:48.253398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.253719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.254007 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.282170 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] MFA Rules not processed for user `06d4d54297384d11acd3142e1f5e714b`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:48.283394 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:48.347482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1412/2842] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:48.352787 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.414682 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3847-eb58-417c-9548-3f85f3d7299a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.419731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.420446 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.420936 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.421235 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.422077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.424003 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.454843 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1431/2843] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:48 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:48.459810 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.470860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138cdb6-51af-4299-ac90-da339bf351ba None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.496697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:48.529503 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.530366 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.530658 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.530943 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.531642 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.533757 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.553666 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1413/2844] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:48 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:48.558301 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.569466 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03445cf4-5196-4dc8-9880-b794445d035a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.574105 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.574767 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.575055 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.575338 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.576069 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.577701 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.588614 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1432/2845] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:48.592884 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.603115 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4ce5-9176-496b-80d1-3624873ec4e5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.607836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.608718 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.609011 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.609297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] PATH_INFO: `/v3/projects/d7f4b6007425445cbafdbfb53e7d01a8/users/4c58be0143384f1e8d1b1a32f24d795b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.624026 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f4b6007425445cbafdbfb53e7d01a8, user_id=4c58be0143384f1e8d1b1a32f24d795b, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.626719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.640864 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1414/2846] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:48 2022] PUT /identity/v3/projects/d7f4b6007425445cbafdbfb53e7d01a8/users/4c58be0143384f1e8d1b1a32f24d795b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:48.644783 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.655672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad5ee-4dc6-4f8b-be12-f8e033147e51 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.710840 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d756fe-1947-441c-b87c-38c4bdaf4f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.711476 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d756fe-1947-441c-b87c-38c4bdaf4f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.711706 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d756fe-1947-441c-b87c-38c4bdaf4f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.711989 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d756fe-1947-441c-b87c-38c4bdaf4f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.712719 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d756fe-1947-441c-b87c-38c4bdaf4f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.714346 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d756fe-1947-441c-b87c-38c4bdaf4f1e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.724281 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1433/2847] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:48.728572 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.738996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e4f3-5d5c-40d3-a0a8-59754be6334e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.744133 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.745013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.745301 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.745587 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] PATH_INFO: `/v3/projects/d7f4b6007425445cbafdbfb53e7d01a8/users/4c58be0143384f1e8d1b1a32f24d795b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.747602 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f4b6007425445cbafdbfb53e7d01a8, user_id=4c58be0143384f1e8d1b1a32f24d795b, role_id=8eeb8d0c8ef34650a10e4e0bfc799e5e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:48.750349 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:48.761068 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1415/2848] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:48 2022] PUT /identity/v3/projects/d7f4b6007425445cbafdbfb53e7d01a8/users/4c58be0143384f1e8d1b1a32f24d795b/roles/8eeb8d0c8ef34650a10e4e0bfc799e5e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:48.771731 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.772077 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.772365 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.778972 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:48.798665 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] MFA Rules not processed for user `4c58be0143384f1e8d1b1a32f24d795b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:48.799873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:48.867532 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1434/2849] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:48.878445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:48.880984 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:48.936682 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cb92-768f-49c7-985f-4b963b7562a0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:48.940303 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:48.940950 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:48.941235 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:48.941516 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:48.999468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:49.000814 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:49.010614 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1416/2850] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:53.179912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.181380 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.194020 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe514a2-e1cd-4f31-a6f8-a7ebd804b254 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.197572 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd7388-dc64-4401-a059-36018f0b208c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.198260 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7388-dc64-4401-a059-36018f0b208c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.198549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7388-dc64-4401-a059-36018f0b208c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.198830 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7388-dc64-4401-a059-36018f0b208c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.207653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd7388-dc64-4401-a059-36018f0b208c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.209745 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd7388-dc64-4401-a059-36018f0b208c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.219062 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1435/2851] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:58.064387 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.103827 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:58.129398 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61b8b5-e5e4-4897-9c14-56d6bb0588e8 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.135014 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.135896 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.136187 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.136478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] PATH_INFO: `/v3/users/06d4d54297384d11acd3142e1f5e714b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.137697 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d4d54297384d11acd3142e1f5e714b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.139645 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.185880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d4d54297384d11acd3142e1f5e714b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.198401 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d4d54297384d11acd3142e1f5e714b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.205038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d4d54297384d11acd3142e1f5e714b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.212631 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d4d54297384d11acd3142e1f5e714b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.213404 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d4d54297384d11acd3142e1f5e714b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.219520 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1417/2852] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:58 2022] DELETE /identity/v3/users/06d4d54297384d11acd3142e1f5e714b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:58.224660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.225650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.225650 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.243156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:58.244351 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.303045 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1436/2853] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:58.313309 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.314771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.373266 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540dad7-f3f5-44af-9e18-d45fcec8edd7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.376996 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.377650 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.377966 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.378249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.387620 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.389062 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.399005 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1418/2854] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:58.925953 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.927950 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.939804 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6450fe3-e4df-496f-af6b-f387436c62ed None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.945168 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.946078 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.946371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.946660 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] PATH_INFO: `/v3/projects/db5e8bb8d8c8484f8b9a0c147f287b4e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.947870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5e8bb8d8c8484f8b9a0c147f287b4e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.950274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.000298 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5e8bb8d8c8484f8b9a0c147f287b4e', 'request_id': 'req-8c278d09-ed3d-4465-9bf4-00ec71f189c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.007981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5e8bb8d8c8484f8b9a0c147f287b4e', 'request_id': 'req-8c278d09-ed3d-4465-9bf4-00ec71f189c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.013570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5e8bb8d8c8484f8b9a0c147f287b4e', 'request_id': 'req-8c278d09-ed3d-4465-9bf4-00ec71f189c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.021156 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:59.025339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] The token cache is being invalidate because project db5e8bb8d8c8484f8b9a0c147f287b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:59.025641 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.026755 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.028771 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1437/2855] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:58 2022] DELETE /identity/v3/projects/db5e8bb8d8c8484f8b9a0c147f287b4e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:59.032857 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.043297 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f4d9b-32f9-4d15-9793-3daf609b8c64 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.094527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:59.102288 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.103189 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.103830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.104144 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] PATH_INFO: `/v3/users/4c58be0143384f1e8d1b1a32f24d795b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.105378 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c58be0143384f1e8d1b1a32f24d795b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.107807 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.155482 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c58be0143384f1e8d1b1a32f24d795b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.167558 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c58be0143384f1e8d1b1a32f24d795b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.174365 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c58be0143384f1e8d1b1a32f24d795b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.182763 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c58be0143384f1e8d1b1a32f24d795b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.183428 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e86a51e4-fd6a-4f3f-8930-1223aee53c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c58be0143384f1e8d1b1a32f24d795b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.192917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1419/2856] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:59 2022] DELETE /identity/v3/users/4c58be0143384f1e8d1b1a32f24d795b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:59.464274 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.465866 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.480014 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c278d09-ed3d-4465-9bf4-00ec71f189c3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.537903 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.538772 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.539058 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.539334 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] PATH_INFO: `/v3/projects/d7f4b6007425445cbafdbfb53e7d01a8` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.540575 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f4b6007425445cbafdbfb53e7d01a8)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.542447 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.587208 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f4b6007425445cbafdbfb53e7d01a8', 'request_id': 'req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.593450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f4b6007425445cbafdbfb53e7d01a8', 'request_id': 'req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.599263 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f4b6007425445cbafdbfb53e7d01a8', 'request_id': 'req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.610616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] The token cache is being invalidate because project d7f4b6007425445cbafdbfb53e7d01a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:59.610911 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.611578 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2eec6870-1191-438a-9c6b-7c9fb18e5cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.613141 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1438/2857] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:59 2022] DELETE /identity/v3/projects/d7f4b6007425445cbafdbfb53e7d01a8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:59.619280 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.619583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.619870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.637152 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:59.638409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.698490 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1420/2858] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:59.756562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.756868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.757155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.776238 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:59.777450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.785398 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1439/2859] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:59.789834 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.801081 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d893ec-2b2a-4bfd-8b71-9c3f7208f170 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.805836 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.806570 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.806860 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.807148 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.807920 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:59.808236 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.809899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.823452 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1421/2860] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:59.828137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.838628 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ccaa-803a-4d4d-ac3f-219ec763998a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.843637 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.844306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.844602 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.844885 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.845596 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.847680 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.874943 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1440/2861] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:59.878986 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.889549 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0b61c-2f74-47bd-8fd1-852ab6755a11 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.944917 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.945580 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.945895 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.946177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.946900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.948795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.957106 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:59.971100 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1422/2862] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:59.975233 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.985262 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa3901-363c-4c26-b8dd-258bf5168d56 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.989761 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.990516 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.991053 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.991333 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.992056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.993642 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.004906 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1441/2863] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:00.009562 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.019557 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae05420-977c-4ecc-8062-79b41f598936 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.025037 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288537-ad2d-45c6-bec0-55beb02a7288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.025721 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288537-ad2d-45c6-bec0-55beb02a7288 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.026038 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288537-ad2d-45c6-bec0-55beb02a7288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.026325 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288537-ad2d-45c6-bec0-55beb02a7288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.027056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e288537-ad2d-45c6-bec0-55beb02a7288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.028643 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e288537-ad2d-45c6-bec0-55beb02a7288 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.038704 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1423/2864] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:00.042712 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.053633 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed87189-6805-4d2b-9dd9-db49599bf6fa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.058509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.059426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.059693 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.059968 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] PATH_INFO: `/v3/projects/fea4e9c1694e4b429b7fac0d4845350d/users/a17c0b7ed2294bf4b8352e8f5364e969/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.075278 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea4e9c1694e4b429b7fac0d4845350d, user_id=a17c0b7ed2294bf4b8352e8f5364e969, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.078081 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.090790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1442/2865] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:00 2022] PUT /identity/v3/projects/fea4e9c1694e4b429b7fac0d4845350d/users/a17c0b7ed2294bf4b8352e8f5364e969/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:00.102052 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.102556 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.102669 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.129527 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] MFA Rules not processed for user `a17c0b7ed2294bf4b8352e8f5364e969`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:00.131423 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.198161 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1424/2866] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:00.209904 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.211739 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.243345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:00.272718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856c7b-ba3d-4226-921f-529ad832d676 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.276306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.276947 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.277232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.277509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.285696 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.287678 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.296295 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1443/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:01.705923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.707445 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.810375 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01640e-d7e6-4d12-96b3-501a4852f2ec None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.815601 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.816262 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.816582 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.816876 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.819794 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.832033 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.833627 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd15ef1-8cc3-48c4-a000-887bf43209c3 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.844226 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1425/2868] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:10.292349 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=14 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:11.840545 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=1,keystone:SELECT=80 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.137018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:35.200339 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed819d6-bc88-40e3-8ba2-241e644196c6 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:35.205499 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:35.206401 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.206718 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.207000 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] PATH_INFO: `/v3/users/a17c0b7ed2294bf4b8352e8f5364e969` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.208193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17c0b7ed2294bf4b8352e8f5364e969)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:35.210149 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:35.255009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17c0b7ed2294bf4b8352e8f5364e969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.255681 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17c0b7ed2294bf4b8352e8f5364e969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.262495 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17c0b7ed2294bf4b8352e8f5364e969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.263123 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17c0b7ed2294bf4b8352e8f5364e969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.270984 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17c0b7ed2294bf4b8352e8f5364e969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.288030 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1444/2869] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:35 2022] DELETE /identity/v3/users/a17c0b7ed2294bf4b8352e8f5364e969 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:35.292790 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.293275 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.293383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.310780 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:35.312553 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:35.371413 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1426/2870] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:35.382093 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:35.383625 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:35.441513 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9fc16-3b39-4a8c-aacf-2d06fa2f8196 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:35.445242 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a9635-d413-425e-b4f8-72f5fd89e815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:35.445935 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a9635-d413-425e-b4f8-72f5fd89e815 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.446232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a9635-d413-425e-b4f8-72f5fd89e815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.446509 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a9635-d413-425e-b4f8-72f5fd89e815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.457129 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a9635-d413-425e-b4f8-72f5fd89e815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:35.458604 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a9635-d413-425e-b4f8-72f5fd89e815 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:35.468374 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1445/2871] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:35.749143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:35.750918 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:35.763317 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf840c4-2b7d-41d2-b2ba-22e54b56790d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:35.768632 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:35.769508 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.769828 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.770263 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] PATH_INFO: `/v3/projects/fea4e9c1694e4b429b7fac0d4845350d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.771478 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea4e9c1694e4b429b7fac0d4845350d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:35.773389 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:35.821446 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea4e9c1694e4b429b7fac0d4845350d', 'request_id': 'req-ad178d99-1016-431e-9914-bd4eff90b6b4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.826987 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea4e9c1694e4b429b7fac0d4845350d', 'request_id': 'req-ad178d99-1016-431e-9914-bd4eff90b6b4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.833919 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea4e9c1694e4b429b7fac0d4845350d', 'request_id': 'req-ad178d99-1016-431e-9914-bd4eff90b6b4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.846907 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] The token cache is being invalidate because project fea4e9c1694e4b429b7fac0d4845350d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:35.847201 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.847832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ad178d99-1016-431e-9914-bd4eff90b6b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:35.849367 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1427/2872] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:35 2022] DELETE /identity/v3/projects/fea4e9c1694e4b429b7fac0d4845350d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:35.854450 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.854762 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.855043 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.873033 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:35.875653 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:35.894345 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.938174 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1446/2873] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:35.947177 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.947597 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.947893 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.966719 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:35.967991 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:35.975850 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1428/2874] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:35.980213 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:35.990232 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc92c28-3045-4bdc-910b-53251e316553 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:35.995161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:35.995853 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:35.996143 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:35.996426 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:35.997161 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:35.997480 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:35.999620 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.011215 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1447/2875] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:36.016041 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.025155 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667283a-b1b6-4e05-bc7c-470f01370a1d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.029880 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.030664 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.030962 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.031249 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.031959 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.033867 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.058292 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1429/2876] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:36 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:36.063061 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.073135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e327b3-cc37-438e-aaed-515d62096b62 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.129561 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.130197 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.130490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.130774 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.131471 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.133379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.153306 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1448/2877] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:36 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:36.157581 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.167800 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155de4a4-f906-4f62-84e5-865103a1b512 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.172529 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.173194 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.173487 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.173870 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.174616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.176207 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.187006 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1430/2878] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:36.190808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.200873 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a2c77-d435-4b48-95c9-cab97ede7f2f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.206307 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493e084-b6ef-4810-80e2-0f307f74cc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.206970 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e084-b6ef-4810-80e2-0f307f74cc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.207261 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e084-b6ef-4810-80e2-0f307f74cc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.207549 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e084-b6ef-4810-80e2-0f307f74cc2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.208311 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493e084-b6ef-4810-80e2-0f307f74cc2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.209944 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493e084-b6ef-4810-80e2-0f307f74cc2e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.220036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1449/2879] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:36.224193 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.234681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07103c-89e4-4e31-a32f-f0e52459c1de admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.239314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.240195 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.240491 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.240771 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] PATH_INFO: `/v3/projects/632a04a9c1294fadaa7dd636439f91dc/users/ea394c6435f341bd8f81b86fe3ed0170/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.258493 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=632a04a9c1294fadaa7dd636439f91dc, user_id=ea394c6435f341bd8f81b86fe3ed0170, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.261141 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.274098 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1431/2880] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:36 2022] PUT /identity/v3/projects/632a04a9c1294fadaa7dd636439f91dc/users/ea394c6435f341bd8f81b86fe3ed0170/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:36.284393 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.284695 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.284983 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.312352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] MFA Rules not processed for user `ea394c6435f341bd8f81b86fe3ed0170`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:36.313577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:36.379107 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1450/2881] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:36.390681 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.392523 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:36.413667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:36.452254 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ff95a-332c-4129-9caa-92ad6fa5bc27 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.456616 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0b65-3a28-45ec-bf60-7eec015a1b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.457278 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0b65-3a28-45ec-bf60-7eec015a1b88 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.457594 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0b65-3a28-45ec-bf60-7eec015a1b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.457899 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0b65-3a28-45ec-bf60-7eec015a1b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.466466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0b65-3a28-45ec-bf60-7eec015a1b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.467890 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0b65-3a28-45ec-bf60-7eec015a1b88 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.476795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1432/2882] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:36.504443 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.524319 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:36.569258 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a9b7-582c-4c36-ae4b-27d3e8ac2f30 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.573914 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.574790 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.575085 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.575371 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] PATH_INFO: `/v3/users/ea394c6435f341bd8f81b86fe3ed0170` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.576579 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea394c6435f341bd8f81b86fe3ed0170)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.578616 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.624226 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea394c6435f341bd8f81b86fe3ed0170'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:36.625072 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea394c6435f341bd8f81b86fe3ed0170'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:36.632135 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea394c6435f341bd8f81b86fe3ed0170'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:36.632799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea394c6435f341bd8f81b86fe3ed0170'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:36.639511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea394c6435f341bd8f81b86fe3ed0170'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:36.657044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1451/2883] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:36 2022] DELETE /identity/v3/users/ea394c6435f341bd8f81b86fe3ed0170 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:36.661514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.661832 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.662117 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.679698 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:36.680881 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:36.741956 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1433/2884] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:36.753590 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:36.755036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:36.814704 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fc94c-fb04-4195-866a-9df22b8037a2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:36.818483 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79065b-5cda-411c-a090-190e7e92d988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:36.819159 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79065b-5cda-411c-a090-190e7e92d988 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:36.819452 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79065b-5cda-411c-a090-190e7e92d988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:36.819735 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79065b-5cda-411c-a090-190e7e92d988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:36.829711 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79065b-5cda-411c-a090-190e7e92d988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:36.831146 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79065b-5cda-411c-a090-190e7e92d988 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:36.840456 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1452/2885] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:37.101143 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.102685 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:37.116811 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b07506-0f9b-484d-9ca2-f9ef37ad2456 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.121795 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.123333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.123641 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.123923 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] PATH_INFO: `/v3/projects/632a04a9c1294fadaa7dd636439f91dc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.126051 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=632a04a9c1294fadaa7dd636439f91dc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.128039 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.173924 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '632a04a9c1294fadaa7dd636439f91dc', 'request_id': 'req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:37.179887 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '632a04a9c1294fadaa7dd636439f91dc', 'request_id': 'req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:37.186299 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '632a04a9c1294fadaa7dd636439f91dc', 'request_id': 'req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:37.199911 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] The token cache is being invalidate because project 632a04a9c1294fadaa7dd636439f91dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:37.200206 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:37.200830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-be5eaaad-87cb-49fd-abc4-f7fd829d0cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:37.203502 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1434/2886] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:37 2022] DELETE /identity/v3/projects/632a04a9c1294fadaa7dd636439f91dc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:37.208672 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.209155 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.209273 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.227448 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:37.228651 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:37.261850 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.290039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1453/2887] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:37.297930 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.298229 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.298514 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.316710 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:37.317935 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:37.326015 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1435/2888] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:37.330870 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.340594 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435f915-f3ab-46e8-9ce2-2e31c689ce3f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.345905 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.346562 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.346855 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.347137 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.347875 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:37.348193 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.349808 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.362154 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1454/2889] 10.222.0.57 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:37.366440 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.376584 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f362d-bf42-4d40-aa32-199ca84d7870 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.381188 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.382046 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.382339 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.382633 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.383314 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.385231 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.411315 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1436/2890] 10.222.0.57 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:37 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:37.415352 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.425511 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24700d89-23cf-409c-8c1a-92531145c81d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.488320 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.488990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.489276 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.489552 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.490269 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.492399 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.512868 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1455/2891] 10.222.0.57 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:37 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:37.518912 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.531362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c9edb-8edb-45fb-bd2e-07d0092a8cde admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.536209 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.536933 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.537221 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.537499 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.538260 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.539854 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.551341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1437/2892] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:37.557313 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.573381 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c87fa-be8c-4429-9079-649ab7cda572 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.578981 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457c122-85e2-4e97-9fff-dcbf8dbff320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.579645 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457c122-85e2-4e97-9fff-dcbf8dbff320 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.579967 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457c122-85e2-4e97-9fff-dcbf8dbff320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.580389 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457c122-85e2-4e97-9fff-dcbf8dbff320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.581364 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457c122-85e2-4e97-9fff-dcbf8dbff320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.583009 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457c122-85e2-4e97-9fff-dcbf8dbff320 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.593431 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1456/2893] 10.222.0.57 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:37.597507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.607988 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a2b2e-88cb-42d6-a467-99bc99d60759 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.612696 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.613605 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.614078 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.614362 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] PATH_INFO: `/v3/projects/4a8030c18a9947e3ad366c57f93b822c/users/182f01dd545c415f87920f17908dbc20/roles/bdd97d4caedc488bac31aec6d879f431` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.616583 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.632356 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8030c18a9947e3ad366c57f93b822c, user_id=182f01dd545c415f87920f17908dbc20, role_id=bdd97d4caedc488bac31aec6d879f431)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.635058 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.648217 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1438/2894] 10.222.0.57 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:37 2022] PUT /identity/v3/projects/4a8030c18a9947e3ad366c57f93b822c/users/182f01dd545c415f87920f17908dbc20/roles/bdd97d4caedc488bac31aec6d879f431 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:37.653250 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f252-ec35-460d-95aa-359fd59e9570 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.653547 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f252-ec35-460d-95aa-359fd59e9570 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.653857 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f252-ec35-460d-95aa-359fd59e9570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.671331 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e779f252-ec35-460d-95aa-359fd59e9570 None None] MFA Rules not processed for user `1ec9e7f1acc8440b837c5cce9d23e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:37.672528 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e779f252-ec35-460d-95aa-359fd59e9570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:37.730768 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1457/2895] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:37.742536 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:37.744237 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:37.801022 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7664f5-81d3-4d98-8ffe-f21c7221164f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:37.804739 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:37.805383 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:37.805686 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:37.805994 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:37.816663 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:37.818121 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:37.827609 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1439/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:38.464438 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc625b-a149-446f-a29f-ed9527d90c55 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:38.464932 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc625b-a149-446f-a29f-ed9527d90c55 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:38.465036 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc625b-a149-446f-a29f-ed9527d90c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:38.491799 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d2bc625b-a149-446f-a29f-ed9527d90c55 None None] MFA Rules not processed for user `182f01dd545c415f87920f17908dbc20`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:38.493259 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d2bc625b-a149-446f-a29f-ed9527d90c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:38.528230 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:38.564571 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1458/2897] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:38.575341 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:38.576740 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:38.637028 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ac846-b551-4886-9939-5df3af17b496 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:38.641136 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:38.641840 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:38.642158 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:38.642414 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:38.651204 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:38.652636 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:38.662031 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1440/2898] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:40.003630 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:40.004140 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:40.004243 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:40.021577 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] MFA Rules not processed for user `182f01dd545c415f87920f17908dbc20`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:40.022796 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:40.029539 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1459/2899] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:48.658029 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:50.017018 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=14 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:30.136712 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:30.145409 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837606b2-b425-456a-9592-a56ebb7446ff service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:30.149050 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26d10743fe844a98ecfcae3974dbce4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:30.150256 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:30.150548 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:30.150830 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:30.152615 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:30.241013 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:30.242468 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777201a-035e-4a4b-8446-01782e614e8d service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:30.262667 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1441/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:32.840990 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:32.852670 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682aed3d-024f-4ff4-ac59-815f25bb1fb0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:32.857744 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:32.858407 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:32.858700 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:32.858985 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:32.860763 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:32.875071 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:32.876503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:32.888256 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1460/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:40.260035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=46 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:42.870276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:42.871377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:42.871377 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:42.882570 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:42.888657 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] MFA Rules not processed for user `182f01dd545c415f87920f17908dbc20`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:42.889901 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:42.896803 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1442/2902] 10.222.0.57 () {58 vars in 957 bytes} [Wed Jun 15 14:58:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:45.094379 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:45.095782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:45.107022 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06d81-4fef-4332-816c-91ff1ce35a2f service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:45.110584 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd292189b741b40b8859bfe9883b4c5ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:45.111219 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:45.111504 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:45.111782 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:45.113477 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:45.124027 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:45.125583 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:45.133497 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1461/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:52.883505 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:55.130067 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:42.126116 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:42.127566 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:42.182044 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:42.197486 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b882d8-b03b-431c-8fab-be5b437031d1 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:42.204169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:42.205769 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c944eca-f440-487c-8e96-c3f845103335 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:42.206039 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:42.206978 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:42.207441 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] PATH_INFO: `/v3/users/182f01dd545c415f87920f17908dbc20` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:42.208887 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182f01dd545c415f87920f17908dbc20)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:42.210897 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:42.211391 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:42.212056 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:42.212348 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:42.212634 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:42.214507 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:42.227733 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:42.229333 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:42.241035 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1443/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:42.264490 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182f01dd545c415f87920f17908dbc20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.265743 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182f01dd545c415f87920f17908dbc20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.273056 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182f01dd545c415f87920f17908dbc20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.273748 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182f01dd545c415f87920f17908dbc20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.281997 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182f01dd545c415f87920f17908dbc20'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.299458 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1462/2905] 10.222.0.57 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:42 2022] DELETE /identity/v3/users/182f01dd545c415f87920f17908dbc20 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:59:42.472713 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:42.483087 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba45159-f7c4-4a12-86d6-5d915332812b service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:42.540577 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec9e7f1acc8440b837c5cce9d23e7c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10e451bce05f4f3c975f7e8a5cfb1440', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:42.541439 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:42.541900 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:42.542192 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] PATH_INFO: `/v3/projects/4a8030c18a9947e3ad366c57f93b822c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:42.543363 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8030c18a9947e3ad366c57f93b822c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:42.545276 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:42.590216 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8030c18a9947e3ad366c57f93b822c', 'request_id': 'req-3e974f08-eb66-4073-ab18-f410107255a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.596465 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8030c18a9947e3ad366c57f93b822c', 'request_id': 'req-3e974f08-eb66-4073-ab18-f410107255a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.602471 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8030c18a9947e3ad366c57f93b822c', 'request_id': 'req-3e974f08-eb66-4073-ab18-f410107255a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.615436 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] The token cache is being invalidate because project 4a8030c18a9947e3ad366c57f93b822c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:42.615725 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.616466 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e974f08-eb66-4073-ab18-f410107255a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:42.618319 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1444/2906] 10.222.0.57 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:42 2022] DELETE /identity/v3/projects/4a8030c18a9947e3ad366c57f93b822c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:52.295503 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:52.612476 dv-cinder-843277-3-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:08.671760 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:08.673207 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:08.742912 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24aa1c2-9f1a-45f6-9ea5-a688dc19adf9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:08.802011 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fddfa14cf721489595e79ffaafef6322', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c210b380e73f4922a6e6c1ecc8e4bc6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:08.802671 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:08.802964 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:08.803251 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:08.805041 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:08.816940 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:08.818546 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d826-a0dd-4095-9ce2-c3f8c4f3eecd service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:08.828470 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1463/2907] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:18.824169 dv-cinder-843277-3-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=58 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}